-- Logs begin at Sun 2022-09-11 14:32:57 UTC, end at Sun 2022-09-11 15:37:59 UTC. -- Sep 11 14:57:19.895635 dv-cinder-839768-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 11 14:57:19.904849 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 11 14:57:19.906951 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Sep 11 14:57:19 2022] *** Sep 11 14:57:19.906951 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 11 14:57:19.906951 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 11 14:57:19.906951 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: nodename: dv-cinder-839768-7-iscsi Sep 11 14:57:19.906951 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: machine: x86_64 Sep 11 14:57:19.906951 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: clock source: unix Sep 11 14:57:19.906951 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: pcre jit disabled Sep 11 14:57:19.906951 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: detected number of CPU cores: 4 Sep 11 14:57:19.906951 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: current working directory: / Sep 11 14:57:19.906951 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: detected binary path: /usr/bin/uwsgi-core Sep 11 14:57:19.906951 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: your processes number limit is 47680 Sep 11 14:57:19.907518 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: your memory page size is 4096 bytes Sep 11 14:57:19.907518 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: detected max file descriptor number: 2048 Sep 11 14:57:19.907518 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: lock engine: pthread robust mutexes Sep 11 14:57:19.907518 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: thunder lock: enabled Sep 11 14:57:19.907518 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 11 14:57:19.907518 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 11 14:57:19.946002 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: Python main interpreter initialized at 0x55ca183629c0 Sep 11 14:57:19.946002 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: python threads support enabled Sep 11 14:57:19.946002 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: your server socket listen backlog is limited to 100 connections Sep 11 14:57:19.946002 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: your mercy for graceful operations on workers is 90 seconds Sep 11 14:57:19.946413 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: mapped 403077 bytes (393 KB) for 2 cores Sep 11 14:57:19.946413 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: *** Operational MODE: preforking *** Sep 11 14:57:19.946697 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: *** uWSGI is running in multiple interpreter mode *** Sep 11 14:57:19.946697 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: spawned uWSGI master process (pid: 81669) Sep 11 14:57:19.946915 dv-cinder-839768-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 11 14:57:19.947680 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: spawned uWSGI worker 1 (pid: 81683, cores: 1) Sep 11 14:57:19.947798 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: spawned uWSGI worker 2 (pid: 81684, cores: 1) Sep 11 14:57:19.947957 dv-cinder-839768-7-iscsi devstack@keystone.service[81669]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 11 14:57:24.152652 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 14:57:24.152652 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: from cryptography.utils import int_from_bytes Sep 11 14:57:24.153526 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 14:57:24.153526 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: from cryptography.utils import int_from_bytes Sep 11 14:57:24.153989 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 14:57:24.153989 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: from cryptography.utils import int_from_bytes Sep 11 14:57:24.154108 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 14:57:24.154108 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: from cryptography.utils import int_from_bytes Sep 11 14:57:24.202823 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 11 14:57:24.203057 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 11 14:57:24.203179 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] command line args: [] {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 11 14:57:24.203420 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 11 14:57:24.203553 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 11 14:57:24.203638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ================================================================================ {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 11 14:57:24.203706 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 11 14:57:24.203776 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] command line args: [] {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 11 14:57:24.203835 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] admin_token = **** {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.204043 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 11 14:57:24.204179 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] config_dir = [] {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.204179 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.204319 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ================================================================================ {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 11 14:57:24.204381 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] config_source = [] {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.204438 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] admin_token = **** {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.204500 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] debug = True {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.204661 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] config_dir = [] {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.204748 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.204998 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.204998 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] config_source = [] {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.205119 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.205305 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] debug = True {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.205410 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.205517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.205651 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.205750 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.205848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.205948 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.206059 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] list_limit = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.206154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.206253 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] log_config_append = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.206253 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.206438 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.206538 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] log_dir = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.206641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] list_limit = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.206737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] log_file = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.206836 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] log_config_append = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.206929 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.207032 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.207032 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] log_dir = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.207220 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.207321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] log_file = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.207439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.207540 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.207641 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.207751 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.207848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.207848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.208033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.208133 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.208227 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.208227 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.208415 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.208509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.208609 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.208704 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.208816 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.208921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.209025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.209123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.209263 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.209368 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.209441 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.209512 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.209581 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.209759 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.209868 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.209980 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.210249 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.210370 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.44/identity {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.210478 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] publish_errors = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.210562 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.210728 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.210981 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.211183 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.44/identity {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.211292 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.211391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] publish_errors = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.211501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.211596 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.211689 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.211781 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.211896 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] standard_threads = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.212005 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.212101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.212195 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.212288 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.212410 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.212533 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.212651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] standard_threads = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.212769 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] use_journal = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.212881 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.212998 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] use_json = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.212998 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] use_stderr = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.213261 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.213339 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] use_syslog = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.213398 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.213518 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.213600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] use_journal = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.213600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] use_json = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.213744 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.213816 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] use_stderr = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.213875 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.213943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] use_syslog = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.214012 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.214087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 14:57:24.214168 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.214240 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.214378 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.214512 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.214647 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.214725 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.214856 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.214934 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.215037 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.215124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.215271 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.215396 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.215518 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.215636 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.215748 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.215866 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.216027 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.216179 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.216307 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.216421 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.216532 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.216652 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.216770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.216893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.217018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.217154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.217271 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.217383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.217497 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.217608 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.217737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.217858 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.217976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.218089 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.218205 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.218408 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.218530 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.218643 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.218753 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.218863 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.218979 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.219104 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.219217 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.219320 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.219435 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.219547 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.219663 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.219783 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.219895 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.220006 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.220117 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.220227 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.220337 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.220456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.220568 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.220679 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.220795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.220912 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.221024 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.221171 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.221285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.221396 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.221507 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.221618 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.221728 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.221851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.221964 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.222075 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.222185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.222295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.222411 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.222532 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.222644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.223113 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.223226 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.223337 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.223453 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.223580 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.223693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.223810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.223921 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.224032 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.224147 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.224336 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.224450 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.224560 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.224670 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.224789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.224904 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.225023 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.225192 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.225305 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.225417 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.225527 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.225650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.225772 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.225884 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.226001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.226118 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.226220 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.226321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.226424 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.226520 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.226613 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.226707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.226800 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.226901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.227002 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.227098 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.227191 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.227319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.227424 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.227556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.227669 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.227781 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.227893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.228004 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.228114 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.228237 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.228357 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.228478 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.228591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.228704 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.228816 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.228935 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.229048 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.229271 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.229391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.229495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.229612 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.229736 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.229841 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.229959 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.230071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.230182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.230293 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.230404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.230515 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.230633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.230752 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.230866 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.230976 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.231088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.231199 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.231321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.231425 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.231535 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.231646 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.231757 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.231867 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.231987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.232101 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.232213 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.232331 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.232443 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.232540 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.232540 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.232726 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.232821 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.232921 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.233035 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.233224 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.233356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.233472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.233585 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.233696 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.233808 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.233920 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.234042 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.234156 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.234268 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.234380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.234492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.234604 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.234724 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.234837 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.234949 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.235061 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.235173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.235285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.235285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.235580 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.235693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.235811 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.235924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.236045 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.236169 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.236291 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.236405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.236517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.236635 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.236748 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.236866 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.236987 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.237101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.237246 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.237362 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.237487 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.237601 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.237724 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.237838 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.237956 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.238182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.238312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.238426 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.238547 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.238669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.238783 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.238901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] auth.external = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.239015 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.239154 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.239281 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.239396 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.239520 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.239635 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.239747 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] auth.external = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.239860 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] auth.password = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.239973 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.240085 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] auth.token = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.240205 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.240323 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.240436 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.240549 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.240661 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] auth.password = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.240773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.240893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] auth.token = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.241008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.241120 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.241273 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.241387 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.241500 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.241626 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.241742 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.241854 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.241967 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] credential.caching = True {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.242079 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.242198 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.242321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.242436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.242549 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.242661 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.242772 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] credential.caching = True {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.242884 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.243005 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.243119 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.243233 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.243345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.243458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.243615 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.243740 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.243855 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.243968 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.244081 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.244194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.244306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.244432 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.244550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.244702 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.244818 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.244931 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.245043 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.245228 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.245347 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.245460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.245574 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.245687 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.245808 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] federation.caching = True {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.245923 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.246036 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.246036 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.246248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.246361 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.246482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.246598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.246710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] federation.caching = True {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.246822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.246936 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.247049 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.247169 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.247955 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.248084 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.248184 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.248279 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.248375 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.248483 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.248584 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.248682 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.248778 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.248933 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.249038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] identity.caching = True {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.249180 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.249313 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.249414 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.249511 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.249608 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.249704 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.249884 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.249985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.250081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] identity.caching = True {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.250177 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.250272 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.250367 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.250474 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.250603 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.250704 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.250801 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.250896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.250998 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.251102 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.251200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.251295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.251391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.251493 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.251589 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.251693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.251791 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.251902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.252000 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.252096 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.252191 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.252300 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.252398 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.252493 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.252588 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.252690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.252786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.252891 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.252989 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.253072 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.253277 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.253393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.253506 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.253633 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.253831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.253831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.254048 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.254161 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.254283 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.254410 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.254526 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.254526 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.254739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.254859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.254982 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.255098 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.255218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.255218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.255432 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.255545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.255666 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.255782 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.255901 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.256016 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.256130 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.256243 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.256368 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.256484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.256598 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.256711 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.256825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.256938 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.257063 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.257205 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.257321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.257434 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.257552 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.257665 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.257792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.257908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.258022 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.258136 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.258250 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.258363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.258485 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.258600 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.258714 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.258828 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.258941 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.259054 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.259176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.259293 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.259407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.259520 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.259642 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.259758 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.259883 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.259998 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.260113 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.260218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.260218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.user = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.260446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.260568 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.260690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.260805 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.260919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.261039 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.261212 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.261343 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.261459 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.261573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.261686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.261686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.user = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.261905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.261905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.262129 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.262242 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.262357 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.262470 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.262583 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.262704 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.262872 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.263008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.263124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.263238 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.263352 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.263476 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.263592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.263706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.263818 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.263932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.264046 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.264169 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.264284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.264399 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.264513 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.264627 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.264740 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.264864 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.264979 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.265092 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.265219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.265333 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.265446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.265574 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.265691 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.265806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.265920 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.266033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.266147 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.266269 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.266385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.266499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.266614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.266728 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.266841 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.266964 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.267079 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.267192 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.267305 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.267419 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.267536 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.267660 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.267775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.267888 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.268002 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.268117 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.268117 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.268338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.268453 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.268571 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.268686 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.268799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.268912 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] resource.caching = True {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.269034 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.269163 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.269278 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.269392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.269507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.269620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.269755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.269871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.269985 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.270099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.270212 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.270334 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.270450 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.270564 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.270677 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] resource.caching = True {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.270777 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.270873 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.270977 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.271076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.271173 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] role.caching = True {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.271269 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.271366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] role.driver = sql {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.271462 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.271644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.271746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.271844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.271940 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.272037 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.272134 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.272238 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.272337 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.272433 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.272529 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.272625 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.272722 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] role.caching = True {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.272832 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.272930 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] role.driver = sql {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.273027 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.273122 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.273270 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.273367 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.273478 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.273578 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.273676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.273771 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.273868 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.273965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.274069 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.274167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.274264 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.274360 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.274360 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.274590 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.274695 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.274792 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.274887 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.274984 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.275080 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.275187 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.275292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.275390 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.275627 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.275755 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.275870 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.275993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.275993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.276206 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.276325 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.276439 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.276590 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.276708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.276822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.276935 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.277050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.277178 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.277307 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.277424 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.277541 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.277698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.277816 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.277930 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.278053 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.278053 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.278270 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.278383 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.278562 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.278681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.278860 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.278987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] token.caching = True {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.279103 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.279217 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.279321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.279437 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.279561 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.279677 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.279791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.279905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.280018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.280131 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.280253 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.280368 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.280481 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.280594 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.280707 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.280811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.280932 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] token.caching = True {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.281047 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.281222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.281339 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.281453 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.281566 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.281691 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.281807 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.281920 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.282034 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.282147 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.282261 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.282383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.282498 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.282620 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.282743 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.282859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.282972 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.283085 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.283207 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] database.connection = **** {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.283323 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.283436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.283549 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.283549 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.283762 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.283762 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.283985 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.284098 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.284211 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.284323 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.284436 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.284562 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.284686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.284809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.284924 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.285037 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.285164 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] database.connection = **** {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.285280 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.285394 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.285521 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.285622 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.285718 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.285831 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.285944 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.286057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.286178 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.286293 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.286406 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.286520 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.286634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.286746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.286868 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.286983 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.287096 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.287209 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.287331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.287331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81683) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 11 14:57:24.287558 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.287558 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.287558 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.287558 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.287558 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.287558 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.287558 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.287558 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.288376 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 14:57:24.288376 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81684) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 11 14:57:24.316776 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.317079 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.317235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.317857 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.318010 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.318138 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.325994 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.326805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.355978 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.356203 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.356982 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.357240 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.413021 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.414342 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.462345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.462568 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.463143 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.463384 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.623249 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.623559 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.623649 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.623870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.624058 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.624307 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.628136 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.628348 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.628606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.628783 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.629012 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.629227 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.648085 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.648302 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.648385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.648480 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.648548 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.648609 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.648739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.648933 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.649032 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.649185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.649268 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.649363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.649450 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.649609 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.661294 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.662152 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.674366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.674548 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.674779 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.674906 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.674990 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.675066 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.675186 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.675277 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.675406 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.675552 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.675632 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81683) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.675706 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.675924 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.676147 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81684) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 14:57:24.692495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.692725 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 11 14:57:24.693025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.693171 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.693300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 11 14:57:24.693450 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 11 14:57:24.693566 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.693684 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.693807 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.693922 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 11 14:57:24.694052 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.694165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.694282 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.694398 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.694514 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.694758 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.704091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.704374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.704645 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.704765 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.704930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.705070 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.705298 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.705581 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.712546 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81684) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.713032 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81683) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.715428 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81684) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.715691 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.716009 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.716176 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81683) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.716364 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.716467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.716638 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.716844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.716979 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.717213 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.717340 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.717499 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.717608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.717709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.717809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.718001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.718173 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.718285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.718385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.718487 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.718772 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.719033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.741022 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.742199 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.742536 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81684) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.742825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.743697 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81683) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.743979 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.747724 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81684) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.748004 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.748284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.748743 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81683) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.749030 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.749339 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.754838 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81684) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.755103 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.755610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81683) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.755904 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.760971 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81684) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.761296 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.761585 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.761754 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81683) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.761853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.762085 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.762229 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.762366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.762448 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.762623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.762761 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.762927 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.763053 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.763220 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.763480 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.763747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.778221 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.778595 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.779258 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.779530 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.782499 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81684) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.783420 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81683) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.785735 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.786032 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.786701 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.787006 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.790185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81684) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.791195 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81683) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.793223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81684) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.794436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81683) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.796393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.796687 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.796966 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.797267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.797543 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.797668 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.797828 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.797914 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.798151 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.798434 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.798710 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.798987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.813289 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81684) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.813587 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.813855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.814118 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.814435 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81683) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.814739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.815009 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.815281 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.820933 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.822067 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.822481 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.823578 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.823965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.824243 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.825048 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.825346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.827037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81684) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.827329 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.827607 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.827891 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.828017 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81683) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.828279 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.828452 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.828580 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.828858 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.829282 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.839331 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81684) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.839625 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.839910 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.840181 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.840348 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81683) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.840564 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.840692 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.840807 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.840897 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.841191 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.841343 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.841604 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.841880 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.842164 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.857022 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81684) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.857813 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81683) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.860001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81684) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.860757 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81683) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.862950 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.863687 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.864488 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.865174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.866024 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81684) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.866310 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.866634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81683) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.866762 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.866961 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.867302 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.873166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.873683 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.874632 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81684) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.875148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81683) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.877561 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.877978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.878104 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.878246 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.878495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.878638 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.878784 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.879045 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.886249 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81684) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.886553 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.886775 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81683) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.886903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.887042 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.887334 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.893639 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81684) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.893926 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.894189 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81683) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 14:57:24.894318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.894483 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.894610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.894724 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.894829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.895024 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.895178 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.895271 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.895348 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.895511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.895615 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.895801 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.895929 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.896048 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.896139 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.896339 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.896464 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.896583 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.896730 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.896847 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.896938 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.897124 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.897272 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81684) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.897426 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.897713 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81683) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 14:57:24.930920 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55ca183629c0 pid: 81684 (default app) Sep 11 14:57:24.931429 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55ca183629c0 pid: 81683 (default app) Sep 11 14:57:24.934336 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb3a9e-f129-4768-bf9e-b9073bf80991 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:24.934616 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb3a9e-f129-4768-bf9e-b9073bf80991 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:24.934948 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb3a9e-f129-4768-bf9e-b9073bf80991 None None] PATH_INFO: `/v3` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:24.939079 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1/1] 10.222.0.44 () {50 vars in 791 bytes} [Sun Sep 11 14:57:24 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:31.713461 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6afe42-6e86-437e-95cc-a7d8a5dcd98d None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:31.714045 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6afe42-6e86-437e-95cc-a7d8a5dcd98d None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:31.714045 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6afe42-6e86-437e-95cc-a7d8a5dcd98d None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:31.718448 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1/2] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:57:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:57:31.725377 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9248c-8d2c-4d94-b73c-d27066a65dc0 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:31.726030 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9248c-8d2c-4d94-b73c-d27066a65dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:31.726030 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9248c-8d2c-4d94-b73c-d27066a65dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:31.796386 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: INFO dbcounter [None req-e1b9248c-8d2c-4d94-b73c-d27066a65dc0 None None] Registered counter for database keystone Sep 11 14:57:31.816124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG oslo_db.sqlalchemy.engines [None req-e1b9248c-8d2c-4d94-b73c-d27066a65dc0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81684) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 11 14:57:31.823206 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writer thread running {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 11 14:57:32.003877 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG passlib.handlers.bcrypt [None req-e1b9248c-8d2c-4d94-b73c-d27066a65dc0 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81684) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 11 14:57:32.004191 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG passlib.handlers.bcrypt [None req-e1b9248c-8d2c-4d94-b73c-d27066a65dc0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81684) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 11 14:57:32.063973 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-e1b9248c-8d2c-4d94-b73c-d27066a65dc0 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:57:32.065654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-e1b9248c-8d2c-4d94-b73c-d27066a65dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:57:32.213055 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 2/3] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:57:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 489 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 11 14:57:32.677797 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6afe42-6e86-437e-95cc-a7d8a5dcd98d None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:32.753014 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: INFO dbcounter [None req-7e6afe42-6e86-437e-95cc-a7d8a5dcd98d None None] Registered counter for database keystone Sep 11 14:57:32.774984 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG oslo_db.sqlalchemy.engines [None req-7e6afe42-6e86-437e-95cc-a7d8a5dcd98d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81683) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 11 14:57:32.782901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writer thread running {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 11 14:57:32.903617 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6afe42-6e86-437e-95cc-a7d8a5dcd98d None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:32.908543 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b6e61-4ed0-4d32-a573-61d399d836fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:32.909219 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b6e61-4ed0-4d32-a573-61d399d836fd admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:32.909498 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b6e61-4ed0-4d32-a573-61d399d836fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:32.909765 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b6e61-4ed0-4d32-a573-61d399d836fd admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:32.919344 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075b6e61-4ed0-4d32-a573-61d399d836fd admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:32.971307 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: WARNING keystone.common.rbac_enforcer.enforcer [None req-075b6e61-4ed0-4d32-a573-61d399d836fd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 11 14:57:32.971498 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075b6e61-4ed0-4d32-a573-61d399d836fd admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:32.982377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 2/4] 10.222.0.44 () {58 vars in 1324 bytes} [Sun Sep 11 14:57:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 14:57:32.988870 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9248c-8d2c-4d94-b73c-d27066a65dc0 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:33.007013 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9248c-8d2c-4d94-b73c-d27066a65dc0 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:33.012562 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7709fe-7f00-4d4b-b3ff-f1718783781b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:33.013737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7709fe-7f00-4d4b-b3ff-f1718783781b admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:33.014019 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7709fe-7f00-4d4b-b3ff-f1718783781b admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:33.014291 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7709fe-7f00-4d4b-b3ff-f1718783781b admin admin] PATH_INFO: `/v3/projects/866dbf4792e0480eb3f18a3e97bbaa75` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:33.015995 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7709fe-7f00-4d4b-b3ff-f1718783781b admin admin] RBAC: Authorizing `identity:get_project(project_id=866dbf4792e0480eb3f18a3e97bbaa75)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:33.069335 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9e7709fe-7f00-4d4b-b3ff-f1718783781b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 11 14:57:33.069676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7709fe-7f00-4d4b-b3ff-f1718783781b admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:33.074258 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 3/5] 10.222.0.44 () {56 vars in 1207 bytes} [Sun Sep 11 14:57:32 2022] GET /identity/v3/projects/866dbf4792e0480eb3f18a3e97bbaa75 => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:35.282536 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42649a02-94b0-41ff-badc-3d3afaa470ab None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:35.283147 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42649a02-94b0-41ff-badc-3d3afaa470ab None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:35.283147 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42649a02-94b0-41ff-badc-3d3afaa470ab None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:35.287291 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 3/6] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:57:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:57:35.293869 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d574fc6-0763-49bf-b30c-3618c239b260 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:35.294492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d574fc6-0763-49bf-b30c-3618c239b260 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:35.294492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d574fc6-0763-49bf-b30c-3618c239b260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:35.316575 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-0d574fc6-0763-49bf-b30c-3618c239b260 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:57:35.318204 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-0d574fc6-0763-49bf-b30c-3618c239b260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:57:35.326950 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 4/7] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:57:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 11 14:57:35.591621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42649a02-94b0-41ff-badc-3d3afaa470ab None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:35.605417 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42649a02-94b0-41ff-badc-3d3afaa470ab None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:35.610439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aba0d5-b996-47cd-b00a-34401130ddd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:35.611145 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aba0d5-b996-47cd-b00a-34401130ddd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:35.611468 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aba0d5-b996-47cd-b00a-34401130ddd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:35.611783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aba0d5-b996-47cd-b00a-34401130ddd4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:35.621543 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aba0d5-b996-47cd-b00a-34401130ddd4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:35.623430 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aba0d5-b996-47cd-b00a-34401130ddd4 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:35.633985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 4/8] 10.222.0.44 () {58 vars in 1324 bytes} [Sun Sep 11 14:57:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 14:57:35.640236 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d574fc6-0763-49bf-b30c-3618c239b260 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:35.650317 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d574fc6-0763-49bf-b30c-3618c239b260 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:35.655499 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38947789-2fca-444d-a618-62d71995b785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:35.656208 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38947789-2fca-444d-a618-62d71995b785 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:35.656525 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38947789-2fca-444d-a618-62d71995b785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:35.656832 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38947789-2fca-444d-a618-62d71995b785 admin admin] PATH_INFO: `/v3/users/e6c6442222c54f34a84fdb4577045a9f` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:35.658371 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38947789-2fca-444d-a618-62d71995b785 admin admin] RBAC: Authorizing `identity:get_user(user_id=e6c6442222c54f34a84fdb4577045a9f)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:35.660360 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38947789-2fca-444d-a618-62d71995b785 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:35.662084 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 5/9] 10.222.0.44 () {56 vars in 1198 bytes} [Sun Sep 11 14:57:35 2022] GET /identity/v3/users/e6c6442222c54f34a84fdb4577045a9f => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:37.943427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7520f-6d86-4e92-85b8-1914c1931c79 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:37.944020 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7520f-6d86-4e92-85b8-1914c1931c79 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:37.944020 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7520f-6d86-4e92-85b8-1914c1931c79 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:37.945345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 5/10] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:57:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:57:37.951305 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff84c4e-0860-49a6-ac9f-21d1cf164ade None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:37.951817 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff84c4e-0860-49a6-ac9f-21d1cf164ade None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:37.951906 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff84c4e-0860-49a6-ac9f-21d1cf164ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:37.974034 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-2ff84c4e-0860-49a6-ac9f-21d1cf164ade None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:57:37.975255 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-2ff84c4e-0860-49a6-ac9f-21d1cf164ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:57:38.002860 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5319b65-b919-47c6-82da-d7ce47d027e8 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:38.003114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5319b65-b919-47c6-82da-d7ce47d027e8 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:38.003385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5319b65-b919-47c6-82da-d7ce47d027e8 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:38.004717 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 6/11] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:57:38.010261 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe7747-324e-4c53-9211-de3de5638e02 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:38.010522 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe7747-324e-4c53-9211-de3de5638e02 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:38.010792 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe7747-324e-4c53-9211-de3de5638e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:38.054485 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG passlib.handlers.bcrypt [None req-45fe7747-324e-4c53-9211-de3de5638e02 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81683) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 11 14:57:38.054879 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG passlib.handlers.bcrypt [None req-45fe7747-324e-4c53-9211-de3de5638e02 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81683) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 11 14:57:38.066561 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 6/12] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:57:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:57:38.104062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-45fe7747-324e-4c53-9211-de3de5638e02 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:57:38.105779 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-45fe7747-324e-4c53-9211-de3de5638e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:57:38.221025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 7/13] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:57:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:57:38.330994 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff84c4e-0860-49a6-ac9f-21d1cf164ade None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:38.360394 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:38.459157 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff84c4e-0860-49a6-ac9f-21d1cf164ade None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:38.495176 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe7747-324e-4c53-9211-de3de5638e02 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:38.496252 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6719a-2f3d-46ec-b103-de5b19ddbb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:38.496946 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6719a-2f3d-46ec-b103-de5b19ddbb48 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:38.497462 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6719a-2f3d-46ec-b103-de5b19ddbb48 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:38.497778 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6719a-2f3d-46ec-b103-de5b19ddbb48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:38.511953 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb6719a-2f3d-46ec-b103-de5b19ddbb48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:57:38.522304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-cfb6719a-2f3d-46ec-b103-de5b19ddbb48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:57:38.522304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:38.522304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:38.522304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:38.522304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:38.522304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:38.522304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:38.522304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:38.522304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:38.522304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:38.522304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:38.522304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:38.522304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:57:38.522304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:57:38.522304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:57:38.522304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:57:38.523407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:57:38.523407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:57:38.523407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:57:38.523407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:57:38.523407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:38.523407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:38.523407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:38.523407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:38.523407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:38.523407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:38.523407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:38.523407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:38.523407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:38.523407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:38.523407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:38.523407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:38.524237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:38.524237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:38.524237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:38.524237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:38.524237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:57:38.524237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:57:38.524237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:57:38.524237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:57:38.524237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:57:38.524237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:57:38.524237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:57:38.524237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:57:38.524237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 7/14] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 14:57:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:38.530713 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6719a-2f3d-46ec-b103-de5b19ddbb48 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:38.609397 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe7747-324e-4c53-9211-de3de5638e02 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:38.612102 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:38.620736 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:38.643610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6719a-2f3d-46ec-b103-de5b19ddbb48 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:38.659142 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ce163-92dc-4bcc-93c2-7081538f266e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:38.660577 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ce163-92dc-4bcc-93c2-7081538f266e None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:38.660972 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ce163-92dc-4bcc-93c2-7081538f266e None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:38.661430 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ce163-92dc-4bcc-93c2-7081538f266e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:38.691154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef252a-fa31-4a8a-8cea-88c553bb2fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:38.691790 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef252a-fa31-4a8a-8cea-88c553bb2fbe None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:38.692069 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef252a-fa31-4a8a-8cea-88c553bb2fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:38.692338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef252a-fa31-4a8a-8cea-88c553bb2fbe None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:38.693021 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef252a-fa31-4a8a-8cea-88c553bb2fbe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:38.693365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef252a-fa31-4a8a-8cea-88c553bb2fbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:38.694785 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef252a-fa31-4a8a-8cea-88c553bb2fbe None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:38.709879 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 8/15] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:57:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:57:38.716027 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef252a-fa31-4a8a-8cea-88c553bb2fbe None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:38.739440 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6ce163-92dc-4bcc-93c2-7081538f266e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:38.741019 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6ce163-92dc-4bcc-93c2-7081538f266e None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:38.827385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef252a-fa31-4a8a-8cea-88c553bb2fbe None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:38.858652 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 8/16] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:57:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 14:57:38.864604 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ce163-92dc-4bcc-93c2-7081538f266e None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:38.868610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:38.873099 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4754f92d-da8e-4c39-882e-b7af9c0fdb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:38.874094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754f92d-da8e-4c39-882e-b7af9c0fdb49 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:38.874455 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754f92d-da8e-4c39-882e-b7af9c0fdb49 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:38.874770 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754f92d-da8e-4c39-882e-b7af9c0fdb49 None admin] PATH_INFO: `/v3/users/e6c6442222c54f34a84fdb4577045a9f` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:38.876103 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4754f92d-da8e-4c39-882e-b7af9c0fdb49 None admin] RBAC: Authorizing `identity:get_user(user_id=e6c6442222c54f34a84fdb4577045a9f)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:38.877741 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4754f92d-da8e-4c39-882e-b7af9c0fdb49 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:38.879414 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 9/17] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:57:38 2022] GET /identity/v3/users/e6c6442222c54f34a84fdb4577045a9f => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:38.885076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4754f92d-da8e-4c39-882e-b7af9c0fdb49 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:38.902039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:38.970767 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ce163-92dc-4bcc-93c2-7081538f266e None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:38.998435 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4754f92d-da8e-4c39-882e-b7af9c0fdb49 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:39.008297 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ea2a7-3dc8-4ca8-a0fe-5ded7aef292b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:39.009176 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ea2a7-3dc8-4ca8-a0fe-5ded7aef292b None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:39.009490 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ea2a7-3dc8-4ca8-a0fe-5ded7aef292b None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:39.009757 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ea2a7-3dc8-4ca8-a0fe-5ded7aef292b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:39.029969 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295ea2a7-3dc8-4ca8-a0fe-5ded7aef292b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:39.031469 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295ea2a7-3dc8-4ca8-a0fe-5ded7aef292b None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:39.035830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d9a99-6810-4a43-82c1-8656ba73b4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:39.036481 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d9a99-6810-4a43-82c1-8656ba73b4c8 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:39.036755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d9a99-6810-4a43-82c1-8656ba73b4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:39.037022 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d9a99-6810-4a43-82c1-8656ba73b4c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:39.038268 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d9a99-6810-4a43-82c1-8656ba73b4c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:39.039745 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d9a99-6810-4a43-82c1-8656ba73b4c8 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:39.041689 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 10/18] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:57:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:57:39.047671 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d9a99-6810-4a43-82c1-8656ba73b4c8 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:39.051178 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-295ea2a7-3dc8-4ca8-a0fe-5ded7aef292b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:57:39.051178 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:39.051178 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:39.051178 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:39.051178 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:39.051178 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:39.051178 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:39.051178 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:39.051178 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:39.051178 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:39.051178 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:39.051178 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:39.051178 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:57:39.051178 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:57:39.051178 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:57:39.051178 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:57:39.052550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:39.052550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:39.052550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:39.052550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:39.052550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:39.052550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:39.052550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:39.052550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:39.052550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:39.052550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:39.052550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:39.052550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:39.052550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:39.052550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:39.052550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:39.052550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:39.052550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:57:39.053830 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:57:39.053830 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:57:39.053830 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:57:39.053830 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 9/19] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:57:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:39.059018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ea2a7-3dc8-4ca8-a0fe-5ded7aef292b None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:39.121091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:39.135236 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:39.155871 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d9a99-6810-4a43-82c1-8656ba73b4c8 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:39.171780 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ea2a7-3dc8-4ca8-a0fe-5ded7aef292b None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:39.192869 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b1c55-f8cf-4d73-942b-719af6c9b9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:39.193514 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b1c55-f8cf-4d73-942b-719af6c9b9e3 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:39.193789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b1c55-f8cf-4d73-942b-719af6c9b9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:39.194058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b1c55-f8cf-4d73-942b-719af6c9b9e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:39.194781 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b1c55-f8cf-4d73-942b-719af6c9b9e3 None admin] RBAC: Adding query filter params (role.id=c426875a15834aa5b122e2a62267d15a, scope.domain.id=default, user.id=e6c6442222c54f34a84fdb4577045a9f) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:39.195082 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b1c55-f8cf-4d73-942b-719af6c9b9e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:39.196528 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b1c55-f8cf-4d73-942b-719af6c9b9e3 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:39.203508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 11/20] 10.222.0.44 () {56 vars in 1322 bytes} [Sun Sep 11 14:57:39 2022] GET /identity/v3/role_assignments?user.id=e6c6442222c54f34a84fdb4577045a9f&scope.domain.id=default&role.id=c426875a15834aa5b122e2a62267d15a => generated 233 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:39.210147 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ebc79-d4b5-4774-8d1c-5d2ce9b25624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:39.211036 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ebc79-d4b5-4774-8d1c-5d2ce9b25624 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:39.211316 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ebc79-d4b5-4774-8d1c-5d2ce9b25624 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:39.211600 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ebc79-d4b5-4774-8d1c-5d2ce9b25624 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:39.212332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ebc79-d4b5-4774-8d1c-5d2ce9b25624 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:39.212869 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ebc79-d4b5-4774-8d1c-5d2ce9b25624 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:39.214498 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ebc79-d4b5-4774-8d1c-5d2ce9b25624 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:39.235791 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 10/21] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:57:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:41.744276 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d977b4a-3446-44d6-b0be-083ad024ab61 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:41.744775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d977b4a-3446-44d6-b0be-083ad024ab61 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:41.744775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d977b4a-3446-44d6-b0be-083ad024ab61 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:41.746123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 12/22] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:57:41.752034 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7eef6-aea2-4568-80c7-8329233b9ce3 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:41.752987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7eef6-aea2-4568-80c7-8329233b9ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:41.753474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7eef6-aea2-4568-80c7-8329233b9ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:41.782564 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-83c7eef6-aea2-4568-80c7-8329233b9ce3 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:57:41.784422 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-83c7eef6-aea2-4568-80c7-8329233b9ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:57:41.879075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 11/23] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:57:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:57:42.311210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399287d8-1196-4349-a447-66dd84086497 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:42.311855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399287d8-1196-4349-a447-66dd84086497 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:42.312329 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399287d8-1196-4349-a447-66dd84086497 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:42.313898 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 13/24] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:57:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:57:42.320104 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5857270-083e-48df-b401-876240ef4f21 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:42.320701 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5857270-083e-48df-b401-876240ef4f21 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:42.321206 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5857270-083e-48df-b401-876240ef4f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:42.348508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-c5857270-083e-48df-b401-876240ef4f21 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:57:42.350140 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-c5857270-083e-48df-b401-876240ef4f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:57:42.364214 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399287d8-1196-4349-a447-66dd84086497 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:42.406529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:42.474597 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 12/25] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:57:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:57:42.583601 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cf7b8-f4ce-4a24-a870-253eaf420d51 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:42.584239 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cf7b8-f4ce-4a24-a870-253eaf420d51 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:42.584717 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cf7b8-f4ce-4a24-a870-253eaf420d51 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:42.586320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 13/26] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:57:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:57:42.591957 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081eef8d-2ca8-4f92-9f18-715468897743 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:42.592099 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399287d8-1196-4349-a447-66dd84086497 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:42.592608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081eef8d-2ca8-4f92-9f18-715468897743 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:42.593080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081eef8d-2ca8-4f92-9f18-715468897743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:42.632137 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-081eef8d-2ca8-4f92-9f18-715468897743 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:57:42.633746 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-081eef8d-2ca8-4f92-9f18-715468897743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:57:42.652550 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971520b6-35c4-4640-9c58-d73e698c7451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:42.655535 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971520b6-35c4-4640-9c58-d73e698c7451 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:42.656040 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971520b6-35c4-4640-9c58-d73e698c7451 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:42.656526 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971520b6-35c4-4640-9c58-d73e698c7451 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:42.690876 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971520b6-35c4-4640-9c58-d73e698c7451 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:57:42.699443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-971520b6-35c4-4640-9c58-d73e698c7451 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:57:42.699443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:42.699443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:42.699443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:42.699443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:42.699443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:42.699443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:42.699443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:42.699443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:42.699443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:42.699443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:42.699443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:42.699443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:57:42.699443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:57:42.699443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:57:42.699443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:57:42.700365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:57:42.700365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:57:42.700365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:57:42.700365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:57:42.700365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:42.700365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:42.700365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:42.700365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:42.700365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:42.700365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:42.700365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:42.700365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:42.700365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:42.700365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:42.700365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:42.700365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:42.701103 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:42.701103 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:42.701103 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:42.701103 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:42.701103 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:57:42.701103 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:57:42.701103 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:57:42.701103 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:57:42.701103 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:57:42.701103 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:57:42.701103 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:57:42.701103 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:57:42.705083 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 14/27] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 14:57:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:42.713193 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971520b6-35c4-4640-9c58-d73e698c7451 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:42.721473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:42.818019 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 14/28] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:57:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:57:42.838452 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081eef8d-2ca8-4f92-9f18-715468897743 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:42.933184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971520b6-35c4-4640-9c58-d73e698c7451 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:42.999400 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c9bb6-e66f-4d11-826b-23ebfa843947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:43.000419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c9bb6-e66f-4d11-826b-23ebfa843947 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:43.000901 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c9bb6-e66f-4d11-826b-23ebfa843947 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:43.001398 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c9bb6-e66f-4d11-826b-23ebfa843947 None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:43.002321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887c9bb6-e66f-4d11-826b-23ebfa843947 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:43.002834 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887c9bb6-e66f-4d11-826b-23ebfa843947 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:43.004452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887c9bb6-e66f-4d11-826b-23ebfa843947 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:43.014020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 15/29] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:57:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:43.022894 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c9bb6-e66f-4d11-826b-23ebfa843947 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:43.047562 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081eef8d-2ca8-4f92-9f18-715468897743 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:43.058468 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:43.119179 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd8420-3246-4c0b-98a4-7db891f410b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:43.119807 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd8420-3246-4c0b-98a4-7db891f410b3 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:43.120091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd8420-3246-4c0b-98a4-7db891f410b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:43.120363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd8420-3246-4c0b-98a4-7db891f410b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:43.140000 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd8420-3246-4c0b-98a4-7db891f410b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:43.141669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd8420-3246-4c0b-98a4-7db891f410b3 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:43.161486 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:43.167672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-3cfd8420-3246-4c0b-98a4-7db891f410b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:57:43.167672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:43.167672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:43.167672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:43.167672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:43.167672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:43.167672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:43.167672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:43.167672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:43.167672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:43.167672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:43.167672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:43.167672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:57:43.167672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:57:43.167672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:57:43.167672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:57:43.168638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:43.168638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:43.168638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:43.168638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:43.168638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:43.168638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:43.168638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:43.168638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:43.168638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:43.168638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:43.168638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:43.168638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:43.168638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:43.168638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:43.168638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:43.168638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:43.168638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:57:43.169533 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:57:43.169533 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:57:43.169533 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:57:43.169533 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 15/30] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:57:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:57:43.173993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd8420-3246-4c0b-98a4-7db891f410b3 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:43.220905 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c9bb6-e66f-4d11-826b-23ebfa843947 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:43.294750 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39636847-74e2-406c-a7f3-a024692fc973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:43.295400 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39636847-74e2-406c-a7f3-a024692fc973 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:43.295674 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39636847-74e2-406c-a7f3-a024692fc973 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:43.295947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39636847-74e2-406c-a7f3-a024692fc973 None admin] PATH_INFO: `/v3/users/e6c6442222c54f34a84fdb4577045a9f` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:43.297157 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39636847-74e2-406c-a7f3-a024692fc973 None admin] RBAC: Authorizing `identity:get_user(user_id=e6c6442222c54f34a84fdb4577045a9f)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:43.298654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39636847-74e2-406c-a7f3-a024692fc973 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:43.300289 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 16/31] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:57:43 2022] GET /identity/v3/users/e6c6442222c54f34a84fdb4577045a9f => generated 247 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:57:43.301102 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39636847-74e2-406c-a7f3-a024692fc973 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:43.312454 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd8420-3246-4c0b-98a4-7db891f410b3 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:43.374994 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f273f56-ec65-4526-b215-1e1c13df79c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:43.375513 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f273f56-ec65-4526-b215-1e1c13df79c9 None admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:43.375793 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f273f56-ec65-4526-b215-1e1c13df79c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:43.376069 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f273f56-ec65-4526-b215-1e1c13df79c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:43.376773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f273f56-ec65-4526-b215-1e1c13df79c9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:43.378260 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f273f56-ec65-4526-b215-1e1c13df79c9 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:43.478613 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39636847-74e2-406c-a7f3-a024692fc973 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:43.512499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 16/32] 10.222.0.44 () {60 vars in 1131 bytes} [Sun Sep 11 14:57:43 2022] POST /identity/v3/roles => generated 225 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:57:43.514002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812ac35-8150-4d12-9788-cf306617d01c None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:43.514293 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812ac35-8150-4d12-9788-cf306617d01c None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:43.514570 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812ac35-8150-4d12-9788-cf306617d01c None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:43.521174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 17/33] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:57:43.521174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7812ac35-8150-4d12-9788-cf306617d01c None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:43.547522 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:43.566680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c6bbb-a2c2-463c-9884-c4ae8068df3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:43.569461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c6bbb-a2c2-463c-9884-c4ae8068df3c None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:43.572345 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c6bbb-a2c2-463c-9884-c4ae8068df3c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:43.573295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c6bbb-a2c2-463c-9884-c4ae8068df3c None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:43.578414 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c6bbb-a2c2-463c-9884-c4ae8068df3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:43.579579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c6bbb-a2c2-463c-9884-c4ae8068df3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:43.586306 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c6bbb-a2c2-463c-9884-c4ae8068df3c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:43.593483 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:43.633057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 17/34] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:57:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:43.633057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226c879-e0a1-4900-9090-b1b7812ed2ee None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:43.633057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226c879-e0a1-4900-9090-b1b7812ed2ee None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:43.633057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226c879-e0a1-4900-9090-b1b7812ed2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:43.655734 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-0226c879-e0a1-4900-9090-b1b7812ed2ee None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:57:43.657016 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-0226c879-e0a1-4900-9090-b1b7812ed2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:57:43.742515 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7812ac35-8150-4d12-9788-cf306617d01c None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:43.782424 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 18/35] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:57:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:57:43.784219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f82fb0-0299-45bb-aa38-8d38e1f71919 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:43.784690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f82fb0-0299-45bb-aa38-8d38e1f71919 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:43.785127 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f82fb0-0299-45bb-aa38-8d38e1f71919 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:43.786728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 19/36] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:57:43.787389 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f82fb0-0299-45bb-aa38-8d38e1f71919 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:43.801153 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38404c-503d-4d40-93cf-a5662805c7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:43.802426 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38404c-503d-4d40-93cf-a5662805c7ac None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:43.802740 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38404c-503d-4d40-93cf-a5662805c7ac None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:43.803210 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38404c-503d-4d40-93cf-a5662805c7ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:43.806723 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a38404c-503d-4d40-93cf-a5662805c7ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:43.806723 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a38404c-503d-4d40-93cf-a5662805c7ac None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:43.808140 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 18/37] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:57:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:43.811707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b475fcd-6037-49ec-bc8c-5e51d80913c0 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:43.811995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b475fcd-6037-49ec-bc8c-5e51d80913c0 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:43.812271 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b475fcd-6037-49ec-bc8c-5e51d80913c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:43.846769 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-0b475fcd-6037-49ec-bc8c-5e51d80913c0 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:57:43.848749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-0b475fcd-6037-49ec-bc8c-5e51d80913c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:57:43.934881 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f82fb0-0299-45bb-aa38-8d38e1f71919 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:43.953219 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 19/38] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:57:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:57:43.954369 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b475fcd-6037-49ec-bc8c-5e51d80913c0 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:43.963083 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:43.973393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b9ae1-1ba2-410c-a4ff-9f537ed74b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:43.974034 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b9ae1-1ba2-410c-a4ff-9f537ed74b53 None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:43.974318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b9ae1-1ba2-410c-a4ff-9f537ed74b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:43.974587 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b9ae1-1ba2-410c-a4ff-9f537ed74b53 None admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:43.975243 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b9ae1-1ba2-410c-a4ff-9f537ed74b53 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:43.976805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b9ae1-1ba2-410c-a4ff-9f537ed74b53 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:43.983820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:44.079308 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b475fcd-6037-49ec-bc8c-5e51d80913c0 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:44.116145 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0cf1b-f924-4a23-b639-1cb712641f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:44.116876 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0cf1b-f924-4a23-b639-1cb712641f05 None admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:44.117176 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0cf1b-f924-4a23-b639-1cb712641f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:44.117468 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0cf1b-f924-4a23-b639-1cb712641f05 None admin] PATH_INFO: `/v3/domains/default/users/e6c6442222c54f34a84fdb4577045a9f/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:44.119760 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b0cf1b-f924-4a23-b639-1cb712641f05 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e6c6442222c54f34a84fdb4577045a9f, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:44.121259 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b0cf1b-f924-4a23-b639-1cb712641f05 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:44.122682 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 20/39] 10.222.0.44 () {60 vars in 1141 bytes} [Sun Sep 11 14:57:43 2022] POST /identity/v3/projects => generated 301 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:57:44.124184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c4b77-662c-47b7-871c-8d88364244e2 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:44.124463 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c4b77-662c-47b7-871c-8d88364244e2 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:44.124741 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c4b77-662c-47b7-871c-8d88364244e2 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:44.129885 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 21/40] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:57:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:57:44.129885 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c4b77-662c-47b7-871c-8d88364244e2 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:44.150667 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 20/41] 10.222.0.44 () {58 vars in 1361 bytes} [Sun Sep 11 14:57:43 2022] PUT /identity/v3/domains/default/users/e6c6442222c54f34a84fdb4577045a9f/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:57:44.152371 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440262c-0902-46b9-99fa-ab5f44ce7f1e None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:44.155516 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440262c-0902-46b9-99fa-ab5f44ce7f1e None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:44.156084 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440262c-0902-46b9-99fa-ab5f44ce7f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:44.187960 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-2440262c-0902-46b9-99fa-ab5f44ce7f1e None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:57:44.189846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-2440262c-0902-46b9-99fa-ab5f44ce7f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:57:44.323436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 21/42] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:57:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:57:44.324738 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440262c-0902-46b9-99fa-ab5f44ce7f1e None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:44.343442 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:44.397369 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c4b77-662c-47b7-871c-8d88364244e2 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:44.445827 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34737b7d-e77e-492a-ab75-28a6b035c9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:44.446460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34737b7d-e77e-492a-ab75-28a6b035c9dc None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:44.446747 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34737b7d-e77e-492a-ab75-28a6b035c9dc None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:44.447025 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34737b7d-e77e-492a-ab75-28a6b035c9dc None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:44.447782 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34737b7d-e77e-492a-ab75-28a6b035c9dc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:44.449372 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34737b7d-e77e-492a-ab75-28a6b035c9dc None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:44.473303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 22/43] 10.222.0.44 () {60 vars in 1131 bytes} [Sun Sep 11 14:57:44 2022] POST /identity/v3/roles => generated 219 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:57:44.487460 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440262c-0902-46b9-99fa-ab5f44ce7f1e None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:44.547980 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f702c3-dc37-4188-be59-1b10e4b04df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:44.549373 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f702c3-dc37-4188-be59-1b10e4b04df5 None admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:44.549969 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f702c3-dc37-4188-be59-1b10e4b04df5 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:44.550628 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f702c3-dc37-4188-be59-1b10e4b04df5 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:44.551901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f702c3-dc37-4188-be59-1b10e4b04df5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:44.554713 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f702c3-dc37-4188-be59-1b10e4b04df5 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:44.580751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 22/44] 10.222.0.44 () {60 vars in 1131 bytes} [Sun Sep 11 14:57:44 2022] POST /identity/v3/roles => generated 223 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:57:44.652962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34737b7d-e77e-492a-ab75-28a6b035c9dc None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:44.812813 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:44.917747 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34737b7d-e77e-492a-ab75-28a6b035c9dc None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:44.974299 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a911dbed-19d4-4814-b996-b3089439c6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:44.975341 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911dbed-19d4-4814-b996-b3089439c6d9 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:44.975816 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911dbed-19d4-4814-b996-b3089439c6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:44.976255 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911dbed-19d4-4814-b996-b3089439c6d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:44.978263 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a911dbed-19d4-4814-b996-b3089439c6d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:44.980845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a911dbed-19d4-4814-b996-b3089439c6d9 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:44.983757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 23/45] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:57:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:44.991858 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f702c3-dc37-4188-be59-1b10e4b04df5 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:45.061603 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:45.099599 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f702c3-dc37-4188-be59-1b10e4b04df5 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:45.134944 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d948e0c-7f1d-4269-847c-0cc8757ed5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:45.135575 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d948e0c-7f1d-4269-847c-0cc8757ed5f3 None admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:45.135861 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d948e0c-7f1d-4269-847c-0cc8757ed5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:45.136131 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d948e0c-7f1d-4269-847c-0cc8757ed5f3 None admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:45.136784 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d948e0c-7f1d-4269-847c-0cc8757ed5f3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:45.138326 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d948e0c-7f1d-4269-847c-0cc8757ed5f3 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:45.170518 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 23/46] 10.222.0.44 () {60 vars in 1141 bytes} [Sun Sep 11 14:57:44 2022] POST /identity/v3/projects => generated 312 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:57:46.439035 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482d765-8a7e-49df-9817-3ee77765fd3a None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:46.439838 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482d765-8a7e-49df-9817-3ee77765fd3a None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:46.440004 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482d765-8a7e-49df-9817-3ee77765fd3a None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:46.441565 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 24/47] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:57:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:57:46.448101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8a64a-dd50-4cc5-b672-ef4299f66983 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:46.448705 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8a64a-dd50-4cc5-b672-ef4299f66983 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:46.448705 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8a64a-dd50-4cc5-b672-ef4299f66983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:46.468102 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-56f8a64a-dd50-4cc5-b672-ef4299f66983 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:57:46.469360 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-56f8a64a-dd50-4cc5-b672-ef4299f66983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:57:46.539881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 24/48] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:57:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:57:46.800177 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482d765-8a7e-49df-9817-3ee77765fd3a None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:46.904153 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482d765-8a7e-49df-9817-3ee77765fd3a None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:46.906673 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:46.940198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa600b-6f75-4100-b0b9-2a948b99c804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:46.940841 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa600b-6f75-4100-b0b9-2a948b99c804 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:46.941118 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa600b-6f75-4100-b0b9-2a948b99c804 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:46.941424 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa600b-6f75-4100-b0b9-2a948b99c804 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:46.954261 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa600b-6f75-4100-b0b9-2a948b99c804 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:57:46.959573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-acfa600b-6f75-4100-b0b9-2a948b99c804 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:57:46.959573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:46.959573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:46.959573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:46.959573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:46.959573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:46.959573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:46.959573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:46.959573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:46.959573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:46.959573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:46.959573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:46.959573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:57:46.959573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:57:46.959573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:57:46.959573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:57:46.961452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:57:46.961452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:57:46.961452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:57:46.961452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:57:46.961452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:46.961452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:46.961452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:46.961452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:46.961452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:46.961452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:46.961452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:46.961452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:46.961452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:46.961452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:46.961452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:46.961452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:46.962845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:46.962845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:46.962845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:46.962845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:46.962845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:57:46.962845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:57:46.962845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:57:46.962845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:57:46.962845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:57:46.962845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:57:46.962845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:57:46.962845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:57:46.962845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 25/49] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 14:57:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:46.969049 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8a64a-dd50-4cc5-b672-ef4299f66983 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:47.022343 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:47.078025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8a64a-dd50-4cc5-b672-ef4299f66983 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:47.114647 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9564a-10ed-487a-b752-d9008d0842dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:47.115262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9564a-10ed-487a-b752-d9008d0842dd None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:47.115535 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9564a-10ed-487a-b752-d9008d0842dd None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:47.115803 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9564a-10ed-487a-b752-d9008d0842dd None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:47.116473 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b9564a-10ed-487a-b752-d9008d0842dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:47.116765 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b9564a-10ed-487a-b752-d9008d0842dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:47.118190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b9564a-10ed-487a-b752-d9008d0842dd None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:47.130338 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 25/50] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:57:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:47.136704 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa600b-6f75-4100-b0b9-2a948b99c804 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:47.249525 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa600b-6f75-4100-b0b9-2a948b99c804 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:47.289896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c28222-b6fc-4920-859b-b8de21d799e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:47.290536 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c28222-b6fc-4920-859b-b8de21d799e6 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:47.290795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c28222-b6fc-4920-859b-b8de21d799e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:47.291079 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c28222-b6fc-4920-859b-b8de21d799e6 None admin] PATH_INFO: `/v3/users/e6c6442222c54f34a84fdb4577045a9f` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:47.292224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c28222-b6fc-4920-859b-b8de21d799e6 None admin] RBAC: Authorizing `identity:get_user(user_id=e6c6442222c54f34a84fdb4577045a9f)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:47.293727 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c28222-b6fc-4920-859b-b8de21d799e6 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:47.295396 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 26/51] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:57:47 2022] GET /identity/v3/users/e6c6442222c54f34a84fdb4577045a9f => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:47.301250 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9564a-10ed-487a-b752-d9008d0842dd None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:47.348906 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9edab8-5c2a-4181-b3e0-d1331a32d5f5 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:47.349120 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9edab8-5c2a-4181-b3e0-d1331a32d5f5 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:47.349382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9edab8-5c2a-4181-b3e0-d1331a32d5f5 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:47.351099 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 27/52] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:57:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:57:47.357062 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634ff5a-0d42-4c32-9086-12d189b1cb07 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:47.357383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634ff5a-0d42-4c32-9086-12d189b1cb07 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:47.357659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634ff5a-0d42-4c32-9086-12d189b1cb07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:47.378178 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-2634ff5a-0d42-4c32-9086-12d189b1cb07 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:57:47.379774 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-2634ff5a-0d42-4c32-9086-12d189b1cb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:57:47.409528 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:47.452956 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 28/53] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:57:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:57:47.515672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9564a-10ed-487a-b752-d9008d0842dd None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:47.528323 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:47.555852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071f51be-9c6f-416b-89e9-983c8da3ab2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:47.556644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f51be-9c6f-416b-89e9-983c8da3ab2c None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:47.557027 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f51be-9c6f-416b-89e9-983c8da3ab2c None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:47.557463 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f51be-9c6f-416b-89e9-983c8da3ab2c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:47.558992 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071f51be-9c6f-416b-89e9-983c8da3ab2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:47.560717 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071f51be-9c6f-416b-89e9-983c8da3ab2c None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:47.562514 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 26/54] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:57:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:47.568984 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2634ff5a-0d42-4c32-9086-12d189b1cb07 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:47.675121 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2634ff5a-0d42-4c32-9086-12d189b1cb07 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:47.709188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5457a7f-3565-4c82-88f4-0e41639b67e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:47.709922 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5457a7f-3565-4c82-88f4-0e41639b67e0 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:47.710300 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5457a7f-3565-4c82-88f4-0e41639b67e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:47.710653 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5457a7f-3565-4c82-88f4-0e41639b67e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:47.711480 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5457a7f-3565-4c82-88f4-0e41639b67e0 None admin] RBAC: Adding query filter params (role.id=c426875a15834aa5b122e2a62267d15a, scope.domain.id=default, user.id=e6c6442222c54f34a84fdb4577045a9f) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:47.711890 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5457a7f-3565-4c82-88f4-0e41639b67e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:47.713454 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5457a7f-3565-4c82-88f4-0e41639b67e0 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:47.714840 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071f51be-9c6f-416b-89e9-983c8da3ab2c None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:47.721146 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 29/55] 10.222.0.44 () {56 vars in 1322 bytes} [Sun Sep 11 14:57:47 2022] GET /identity/v3/role_assignments?user.id=e6c6442222c54f34a84fdb4577045a9f&scope.domain.id=default&role.id=c426875a15834aa5b122e2a62267d15a => generated 530 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:47.833310 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071f51be-9c6f-416b-89e9-983c8da3ab2c None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:47.871564 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a281c1b4-ede4-410b-8533-028d1186e178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:47.872255 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281c1b4-ede4-410b-8533-028d1186e178 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:47.872546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281c1b4-ede4-410b-8533-028d1186e178 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:47.872824 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281c1b4-ede4-410b-8533-028d1186e178 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:47.874196 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a281c1b4-ede4-410b-8533-028d1186e178 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:47.875829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a281c1b4-ede4-410b-8533-028d1186e178 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:47.877607 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 27/56] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:57:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:57:47.883724 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5457a7f-3565-4c82-88f4-0e41639b67e0 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:47.916796 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:48.005332 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5457a7f-3565-4c82-88f4-0e41639b67e0 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:48.039959 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eabfcf-7270-4b5f-9a97-9de2e0700ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:48.040582 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eabfcf-7270-4b5f-9a97-9de2e0700ed3 None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:48.040862 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eabfcf-7270-4b5f-9a97-9de2e0700ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:48.041128 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eabfcf-7270-4b5f-9a97-9de2e0700ed3 None admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:48.041810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eabfcf-7270-4b5f-9a97-9de2e0700ed3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:48.043286 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eabfcf-7270-4b5f-9a97-9de2e0700ed3 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:48.071433 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 30/57] 10.222.0.44 () {60 vars in 1140 bytes} [Sun Sep 11 14:57:47 2022] POST /identity/v3/projects => generated 298 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:57:50.244698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1811ef-08c4-4822-b538-235f0f58a6c6 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:50.245290 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1811ef-08c4-4822-b538-235f0f58a6c6 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:50.245290 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1811ef-08c4-4822-b538-235f0f58a6c6 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:50.246609 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 28/58] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:57:50.252597 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28def5-f501-4663-9d95-8d384ec371d5 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:50.253088 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28def5-f501-4663-9d95-8d384ec371d5 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:50.253195 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28def5-f501-4663-9d95-8d384ec371d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:50.272702 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-2f28def5-f501-4663-9d95-8d384ec371d5 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:57:50.273964 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-2f28def5-f501-4663-9d95-8d384ec371d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:57:50.346312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 31/59] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:57:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:57:50.607791 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1811ef-08c4-4822-b538-235f0f58a6c6 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:50.664673 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:50.749991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1811ef-08c4-4822-b538-235f0f58a6c6 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:50.785745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12bc24-6bca-4a0f-b49c-3e22a54cf53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:50.786499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12bc24-6bca-4a0f-b49c-3e22a54cf53d None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:50.786767 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12bc24-6bca-4a0f-b49c-3e22a54cf53d None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:50.787046 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12bc24-6bca-4a0f-b49c-3e22a54cf53d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:50.788545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12bc24-6bca-4a0f-b49c-3e22a54cf53d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:50.790261 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12bc24-6bca-4a0f-b49c-3e22a54cf53d None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:50.792030 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 29/60] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:57:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:50.798014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f28def5-f501-4663-9d95-8d384ec371d5 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:50.808423 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:50.904257 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f28def5-f501-4663-9d95-8d384ec371d5 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:50.943093 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb0b405-b726-48f1-a1ec-30accdc32092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:50.943719 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0b405-b726-48f1-a1ec-30accdc32092 None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:50.943998 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0b405-b726-48f1-a1ec-30accdc32092 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:50.944273 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0b405-b726-48f1-a1ec-30accdc32092 None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:50.944955 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb0b405-b726-48f1-a1ec-30accdc32092 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:50.946496 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb0b405-b726-48f1-a1ec-30accdc32092 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:50.976782 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 32/61] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 14:57:50 2022] POST /identity/v3/users => generated 275 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 14:57:54.267738 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b36a5-dbe3-450e-b616-5412393616dd None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:54.268805 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b36a5-dbe3-450e-b616-5412393616dd None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:54.269395 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b36a5-dbe3-450e-b616-5412393616dd None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:54.270962 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 30/62] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:57:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:57:54.278005 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca5602-1018-41fd-b5c9-f2b6b2f1b75e None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:54.278908 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca5602-1018-41fd-b5c9-f2b6b2f1b75e None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:54.279385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca5602-1018-41fd-b5c9-f2b6b2f1b75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:54.307024 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-76ca5602-1018-41fd-b5c9-f2b6b2f1b75e None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:57:54.309502 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-76ca5602-1018-41fd-b5c9-f2b6b2f1b75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:57:54.453438 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:54.468006 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 33/63] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:57:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:57:54.622141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70192364-f6c5-4b7d-aa77-49212b58da2f None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:54.623081 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70192364-f6c5-4b7d-aa77-49212b58da2f None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:54.623803 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70192364-f6c5-4b7d-aa77-49212b58da2f None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:54.626489 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 31/64] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:57:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:57:54.634765 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e566818-4329-4ce9-b3fc-b40f0a6767b0 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:54.635376 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e566818-4329-4ce9-b3fc-b40f0a6767b0 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:54.635872 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e566818-4329-4ce9-b3fc-b40f0a6767b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:54.663887 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-8e566818-4329-4ce9-b3fc-b40f0a6767b0 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:57:54.665608 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-8e566818-4329-4ce9-b3fc-b40f0a6767b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:57:54.767821 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 34/65] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:57:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:57:54.769040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4209482-3a15-4cf6-bcd5-fa6a7da7f7e0 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:54.769565 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4209482-3a15-4cf6-bcd5-fa6a7da7f7e0 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:54.769839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4209482-3a15-4cf6-bcd5-fa6a7da7f7e0 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:54.771179 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 32/66] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:57:54.780770 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314bb42-02e7-4e41-8d99-e98395b7f51e None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:54.781287 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314bb42-02e7-4e41-8d99-e98395b7f51e None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:54.781777 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314bb42-02e7-4e41-8d99-e98395b7f51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:54.819200 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-e314bb42-02e7-4e41-8d99-e98395b7f51e None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:57:54.821058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-e314bb42-02e7-4e41-8d99-e98395b7f51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:57:54.953415 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 35/67] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:57:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:57:54.983742 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4209482-3a15-4cf6-bcd5-fa6a7da7f7e0 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:55.070088 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314bb42-02e7-4e41-8d99-e98395b7f51e None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:55.141432 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4209482-3a15-4cf6-bcd5-fa6a7da7f7e0 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:55.159542 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:55.183051 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000dd88-5cf0-459b-953f-34eb9a272ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:55.183700 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000dd88-5cf0-459b-953f-34eb9a272ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:55.183975 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000dd88-5cf0-459b-953f-34eb9a272ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:55.184248 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000dd88-5cf0-459b-953f-34eb9a272ee7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:55.205040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e000dd88-5cf0-459b-953f-34eb9a272ee7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:57:55.211695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-e000dd88-5cf0-459b-953f-34eb9a272ee7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:57:55.211695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:55.211695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:55.211695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:55.211695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:55.211695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:55.211695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:55.211695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:55.211695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:55.211695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:55.211695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:55.211695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:55.211695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:57:55.211695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:57:55.211695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:57:55.211695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:57:55.212582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:57:55.212582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:57:55.212582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:57:55.212582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:57:55.212582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:55.212582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:55.212582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:55.212582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:55.212582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:55.212582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:55.212582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:55.212582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:55.212582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:55.212582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:55.212582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:55.212582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:55.213404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:55.213404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:55.213404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:55.213404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:55.213404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:57:55.213404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:57:55.213404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:57:55.213404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:57:55.213404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:57:55.213404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:57:55.213404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:57:55.213404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:57:55.216174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 33/68] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:57:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:55.221338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:55.223079 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000dd88-5cf0-459b-953f-34eb9a272ee7 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:55.329676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314bb42-02e7-4e41-8d99-e98395b7f51e None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:55.385522 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000dd88-5cf0-459b-953f-34eb9a272ee7 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:55.397679 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6a3b6-741c-4a91-8459-78a5544e2d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:55.398737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6a3b6-741c-4a91-8459-78a5544e2d1b None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:55.399214 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6a3b6-741c-4a91-8459-78a5544e2d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:55.399677 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6a3b6-741c-4a91-8459-78a5544e2d1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:55.425250 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b6a3b6-741c-4a91-8459-78a5544e2d1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:57:55.429605 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b7f1b-76d4-4e90-9654-eec4445c8441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:55.430636 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b7f1b-76d4-4e90-9654-eec4445c8441 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:55.431166 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b7f1b-76d4-4e90-9654-eec4445c8441 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:55.431662 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b7f1b-76d4-4e90-9654-eec4445c8441 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:55.432672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b7f1b-76d4-4e90-9654-eec4445c8441 None admin] RBAC: Adding query filter params (name=member) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:55.433329 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b7f1b-76d4-4e90-9654-eec4445c8441 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:55.434789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-e9b6a3b6-741c-4a91-8459-78a5544e2d1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:57:55.434789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:55.434789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:55.434789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:55.434789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:55.434789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:55.434789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:55.434789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:55.434789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:55.434789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:55.434789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:55.434789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:55.434789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:57:55.434789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:57:55.434789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:57:55.434789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:57:55.435655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:57:55.435655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:57:55.435655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:57:55.435655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:57:55.435655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:55.435655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:55.435655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:55.435655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:55.435655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:55.435655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:55.435655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:55.435655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:55.435655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:55.435655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:55.435655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:55.435655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:55.436447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:55.436447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:55.436447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:55.436447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:55.436447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:57:55.436447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:57:55.436447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:57:55.436447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:57:55.436447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:57:55.436447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:57:55.436447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:57:55.436447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:57:55.437028 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 36/69] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 14:57:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:57:55.438267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6a3b6-741c-4a91-8459-78a5544e2d1b None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:55.438778 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b7f1b-76d4-4e90-9654-eec4445c8441 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:55.452185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 34/70] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 14:57:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:55.453582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b7f1b-76d4-4e90-9654-eec4445c8441 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:55.500384 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:55.570354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b7f1b-76d4-4e90-9654-eec4445c8441 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:55.608985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3dfa11-0d65-4851-9497-a55fc6f64387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:55.609649 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3dfa11-0d65-4851-9497-a55fc6f64387 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:55.609933 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3dfa11-0d65-4851-9497-a55fc6f64387 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:55.610209 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3dfa11-0d65-4851-9497-a55fc6f64387 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:55.610893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3dfa11-0d65-4851-9497-a55fc6f64387 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:55.611200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3dfa11-0d65-4851-9497-a55fc6f64387 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:55.612638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3dfa11-0d65-4851-9497-a55fc6f64387 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:55.626695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 35/71] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:57:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:55.626695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3dfa11-0d65-4851-9497-a55fc6f64387 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:55.649424 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6a3b6-741c-4a91-8459-78a5544e2d1b None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:55.676180 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:55.711013 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba850f-fa21-428b-82f6-0e34302fb420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:55.712140 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba850f-fa21-428b-82f6-0e34302fb420 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:55.712606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba850f-fa21-428b-82f6-0e34302fb420 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:55.713067 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba850f-fa21-428b-82f6-0e34302fb420 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:55.732997 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba850f-fa21-428b-82f6-0e34302fb420 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:57:55.734037 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:55.738733 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3dfa11-0d65-4851-9497-a55fc6f64387 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:55.743623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-38ba850f-fa21-428b-82f6-0e34302fb420 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:57:55.743623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:55.743623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:55.743623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:55.743623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:55.743623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:55.743623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:55.743623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:55.743623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:55.743623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:55.743623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:55.743623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:55.743623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:57:55.743623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:57:55.743623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:57:55.743623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:57:55.744511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:57:55.744511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:57:55.744511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:57:55.744511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:57:55.744511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:55.744511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:55.744511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:55.744511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:55.744511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:55.744511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:55.744511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:55.744511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:55.744511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:55.744511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:55.744511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:55.744511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:55.745769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:55.745769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:55.745769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:55.745769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:55.745769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:57:55.745769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:57:55.745769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:57:55.745769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:57:55.745769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:57:55.745769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:57:55.745769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:57:55.745769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:57:55.745769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 37/72] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 14:57:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:55.748175 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025749a-52b4-492b-8747-59a344f29f81 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:55.748623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025749a-52b4-492b-8747-59a344f29f81 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:55.749104 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025749a-52b4-492b-8747-59a344f29f81 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:55.757021 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 38/73] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:57:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:57:55.757021 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025749a-52b4-492b-8747-59a344f29f81 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:55.786786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a615f6-5e6d-4603-806a-1970e2e7cac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:55.787558 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a615f6-5e6d-4603-806a-1970e2e7cac3 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:55.787915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a615f6-5e6d-4603-806a-1970e2e7cac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:55.788197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a615f6-5e6d-4603-806a-1970e2e7cac3 None admin] PATH_INFO: `/v3/users/9c5fead1d17444a69ba60c5cf671f4c0` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:55.805787 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a615f6-5e6d-4603-806a-1970e2e7cac3 None admin] RBAC: Authorizing `identity:get_user(user_id=9c5fead1d17444a69ba60c5cf671f4c0)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:55.807256 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a615f6-5e6d-4603-806a-1970e2e7cac3 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:55.809205 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 36/74] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:57:55 2022] GET /identity/v3/users/9c5fead1d17444a69ba60c5cf671f4c0 => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:55.814378 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a615f6-5e6d-4603-806a-1970e2e7cac3 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:55.928630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a615f6-5e6d-4603-806a-1970e2e7cac3 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:55.932102 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025749a-52b4-492b-8747-59a344f29f81 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:55.969905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97922d-bc59-40c5-8c69-5dd842dcf8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:55.970584 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97922d-bc59-40c5-8c69-5dd842dcf8a5 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:55.970851 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97922d-bc59-40c5-8c69-5dd842dcf8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:55.971120 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97922d-bc59-40c5-8c69-5dd842dcf8a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:55.971807 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97922d-bc59-40c5-8c69-5dd842dcf8a5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:55.972110 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97922d-bc59-40c5-8c69-5dd842dcf8a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:55.974071 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97922d-bc59-40c5-8c69-5dd842dcf8a5 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:55.980483 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ba9d4-b223-4889-952a-67a82392d546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:55.981210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ba9d4-b223-4889-952a-67a82392d546 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:55.981571 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ba9d4-b223-4889-952a-67a82392d546 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:55.981846 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ba9d4-b223-4889-952a-67a82392d546 None admin] PATH_INFO: `/v3/users/e6c6442222c54f34a84fdb4577045a9f` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:55.983188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526ba9d4-b223-4889-952a-67a82392d546 None admin] RBAC: Authorizing `identity:get_user(user_id=e6c6442222c54f34a84fdb4577045a9f)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:55.983423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 37/75] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:57:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:55.984644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645c0c0-9aa0-41f4-bc07-e0571ff03d63 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:55.984827 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526ba9d4-b223-4889-952a-67a82392d546 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:55.984973 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645c0c0-9aa0-41f4-bc07-e0571ff03d63 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:55.985298 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645c0c0-9aa0-41f4-bc07-e0571ff03d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:55.992488 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 39/76] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:57:55 2022] GET /identity/v3/users/e6c6442222c54f34a84fdb4577045a9f => generated 247 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:55.992488 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ba9d4-b223-4889-952a-67a82392d546 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:56.009001 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-6645c0c0-9aa0-41f4-bc07-e0571ff03d63 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:57:56.010165 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-6645c0c0-9aa0-41f4-bc07-e0571ff03d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:57:56.024296 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:56.039185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:56.082851 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 38/77] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:57:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:57:56.093372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fe0fd-fd1f-4a50-9924-6adb1846c8e9 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:56.093372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fe0fd-fd1f-4a50-9924-6adb1846c8e9 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:56.093372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fe0fd-fd1f-4a50-9924-6adb1846c8e9 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:56.093372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 39/78] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:57:56.093372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08fe0fd-fd1f-4a50-9924-6adb1846c8e9 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:56.116286 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ba9d4-b223-4889-952a-67a82392d546 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:56.154615 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d74ea0-3e44-4365-b50c-3ff17c43f1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:56.155237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d74ea0-3e44-4365-b50c-3ff17c43f1aa None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:56.155515 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d74ea0-3e44-4365-b50c-3ff17c43f1aa None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:56.155816 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d74ea0-3e44-4365-b50c-3ff17c43f1aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:56.220344 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08fe0fd-fd1f-4a50-9924-6adb1846c8e9 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:56.242075 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d74ea0-3e44-4365-b50c-3ff17c43f1aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:56.243809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d74ea0-3e44-4365-b50c-3ff17c43f1aa None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:56.267365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093aa5e-33f5-4fc8-b829-472918f204f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:56.268188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093aa5e-33f5-4fc8-b829-472918f204f4 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:56.268594 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093aa5e-33f5-4fc8-b829-472918f204f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:56.268983 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093aa5e-33f5-4fc8-b829-472918f204f4 None admin] PATH_INFO: `/v3/users/9c5fead1d17444a69ba60c5cf671f4c0` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:56.270605 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093aa5e-33f5-4fc8-b829-472918f204f4 None admin] RBAC: Authorizing `identity:get_user(user_id=9c5fead1d17444a69ba60c5cf671f4c0)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:56.275326 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093aa5e-33f5-4fc8-b829-472918f204f4 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:56.278359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 40/79] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:57:56 2022] GET /identity/v3/users/9c5fead1d17444a69ba60c5cf671f4c0 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:56.282678 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093aa5e-33f5-4fc8-b829-472918f204f4 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:56.303453 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:56.307676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:56.370694 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 40/80] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:57:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:57:56.372038 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5156bdc-bdf6-4fbf-9690-e84cef3b39ee None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:56.373189 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5156bdc-bdf6-4fbf-9690-e84cef3b39ee None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:56.373189 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5156bdc-bdf6-4fbf-9690-e84cef3b39ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:56.392501 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-f5156bdc-bdf6-4fbf-9690-e84cef3b39ee None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:57:56.393769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f5156bdc-bdf6-4fbf-9690-e84cef3b39ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:57:56.406349 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093aa5e-33f5-4fc8-b829-472918f204f4 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:56.445528 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd0afb-e46b-4b51-9089-d19151d92a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:56.446273 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd0afb-e46b-4b51-9089-d19151d92a0f None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:56.446660 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd0afb-e46b-4b51-9089-d19151d92a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:56.447018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd0afb-e46b-4b51-9089-d19151d92a0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:56.468505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 41/81] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:57:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:57:56.469522 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5156bdc-bdf6-4fbf-9690-e84cef3b39ee None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:56.549026 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:56.567918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd0afb-e46b-4b51-9089-d19151d92a0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:56.570682 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd0afb-e46b-4b51-9089-d19151d92a0f None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:56.582995 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5156bdc-bdf6-4fbf-9690-e84cef3b39ee None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:56.590544 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:56.628219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbeb256-1cf1-4817-b3b8-9714620426c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:56.628892 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbeb256-1cf1-4817-b3b8-9714620426c0 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:56.629188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbeb256-1cf1-4817-b3b8-9714620426c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:56.629464 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbeb256-1cf1-4817-b3b8-9714620426c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:56.706850 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 41/82] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:57:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:57:56.708658 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd0afb-e46b-4b51-9089-d19151d92a0f None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:56.716422 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbeb256-1cf1-4817-b3b8-9714620426c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:56.718226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbeb256-1cf1-4817-b3b8-9714620426c0 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:56.798977 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:56.838147 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 42/83] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:57:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:57:56.838880 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbeb256-1cf1-4817-b3b8-9714620426c0 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:56.869335 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd0afb-e46b-4b51-9089-d19151d92a0f None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:56.886696 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:56.906778 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47399e51-ca19-4957-9a7e-1db7856a1a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:56.907426 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47399e51-ca19-4957-9a7e-1db7856a1a96 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:56.907820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47399e51-ca19-4957-9a7e-1db7856a1a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:56.908264 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47399e51-ca19-4957-9a7e-1db7856a1a96 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:56.920772 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47399e51-ca19-4957-9a7e-1db7856a1a96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:57:56.925634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-47399e51-ca19-4957-9a7e-1db7856a1a96 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:57:56.925634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:57:56.925634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:57:56.925634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:57:56.925634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:57:56.925634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:57:56.925634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:57:56.925634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:57:56.925634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:57:56.925634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:57:56.925634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:57:56.925634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:57:56.925634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:57:56.925634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:57:56.925634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:57:56.925634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:57:56.927040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:57:56.927040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:57:56.927040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:57:56.927040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:57:56.927040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:57:56.927040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:57:56.927040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:57:56.927040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:57:56.927040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:57:56.927040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:57:56.927040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:57:56.927040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:57:56.927040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:57:56.927040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:57:56.927040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:57:56.927040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:57:56.928002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:57:56.928002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:57:56.928002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:57:56.928002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:57:56.928002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:57:56.928002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:57:56.928002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:57:56.928002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:57:56.928002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:57:56.928002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:57:56.928002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:57:56.928002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:57:56.928002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 42/84] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:57:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:57:56.928002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47399e51-ca19-4957-9a7e-1db7856a1a96 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:56.948660 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbeb256-1cf1-4817-b3b8-9714620426c0 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:56.991580 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b5a2cb-b246-4ed9-b08a-46df74e14b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:56.992553 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5a2cb-b246-4ed9-b08a-46df74e14b25 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:56.992914 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5a2cb-b246-4ed9-b08a-46df74e14b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:56.993310 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5a2cb-b246-4ed9-b08a-46df74e14b25 None admin] PATH_INFO: `/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:56.994499 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b5a2cb-b246-4ed9-b08a-46df74e14b25 None admin] RBAC: Authorizing `identity:get_project(project_id=b4cfdc9ec6a74df880eb9dcd5e97b98c)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:56.996081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b5a2cb-b246-4ed9-b08a-46df74e14b25 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:56.997862 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 43/85] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:57:56 2022] GET /identity/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:57.003085 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b5a2cb-b246-4ed9-b08a-46df74e14b25 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:57.042502 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:57.047691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47399e51-ca19-4957-9a7e-1db7856a1a96 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:57.083998 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0b598-6f8c-42f9-bf90-069f0dd140eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:57.084648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0b598-6f8c-42f9-bf90-069f0dd140eb None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:57.084921 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0b598-6f8c-42f9-bf90-069f0dd140eb None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:57.085236 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0b598-6f8c-42f9-bf90-069f0dd140eb None admin] PATH_INFO: `/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:57.086395 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f0b598-6f8c-42f9-bf90-069f0dd140eb None admin] RBAC: Authorizing `identity:get_project(project_id=b4cfdc9ec6a74df880eb9dcd5e97b98c)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:57.087890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f0b598-6f8c-42f9-bf90-069f0dd140eb None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:57.089603 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 43/86] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:57:56 2022] GET /identity/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:57.090314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0b598-6f8c-42f9-bf90-069f0dd140eb None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:57.113857 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b5a2cb-b246-4ed9-b08a-46df74e14b25 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:57.141377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:57.153547 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdb278-10e1-40ed-85de-5dd8cd74ffbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:57.154208 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdb278-10e1-40ed-85de-5dd8cd74ffbe None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:57.154498 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdb278-10e1-40ed-85de-5dd8cd74ffbe None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:57.154772 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdb278-10e1-40ed-85de-5dd8cd74ffbe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:57.156044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cdb278-10e1-40ed-85de-5dd8cd74ffbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:57.157589 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cdb278-10e1-40ed-85de-5dd8cd74ffbe None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:57.159261 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 44/87] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:57:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:57.160051 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdb278-10e1-40ed-85de-5dd8cd74ffbe None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:57.208461 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0b598-6f8c-42f9-bf90-069f0dd140eb None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:57.243081 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad719ab0-e049-4c7d-ace0-307d979c6d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:57.243745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad719ab0-e049-4c7d-ace0-307d979c6d8b None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:57.244015 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad719ab0-e049-4c7d-ace0-307d979c6d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:57.244284 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad719ab0-e049-4c7d-ace0-307d979c6d8b None admin] PATH_INFO: `/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:57.245545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad719ab0-e049-4c7d-ace0-307d979c6d8b None admin] RBAC: Authorizing `identity:get_project(project_id=b4cfdc9ec6a74df880eb9dcd5e97b98c)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:57.247123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad719ab0-e049-4c7d-ace0-307d979c6d8b None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:57.248894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 44/88] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:57:57 2022] GET /identity/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:57.249757 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad719ab0-e049-4c7d-ace0-307d979c6d8b None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:57.272314 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdb278-10e1-40ed-85de-5dd8cd74ffbe None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:57.283785 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:57.314460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b03fa-ba94-4241-b1b4-12993606b950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:57.315312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b03fa-ba94-4241-b1b4-12993606b950 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:57.315655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b03fa-ba94-4241-b1b4-12993606b950 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:57.315995 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b03fa-ba94-4241-b1b4-12993606b950 None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:57.316850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b03fa-ba94-4241-b1b4-12993606b950 None admin] RBAC: Adding query filter params (name=member) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:57.317368 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b03fa-ba94-4241-b1b4-12993606b950 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:57.319130 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b03fa-ba94-4241-b1b4-12993606b950 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:57.329419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 45/89] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 14:57:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:57.330323 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b03fa-ba94-4241-b1b4-12993606b950 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:57.369530 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad719ab0-e049-4c7d-ace0-307d979c6d8b None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:57.384861 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:57.409318 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01189288-2268-405e-8676-365ec1a17194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:57.409985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01189288-2268-405e-8676-365ec1a17194 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:57.410229 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01189288-2268-405e-8676-365ec1a17194 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:57.410502 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01189288-2268-405e-8676-365ec1a17194 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:57.411272 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01189288-2268-405e-8676-365ec1a17194 None admin] RBAC: Adding query filter params (role.id=3e3756d9482f4b9b86271eacdc19f523, scope.project.id=b4cfdc9ec6a74df880eb9dcd5e97b98c, user.id=9c5fead1d17444a69ba60c5cf671f4c0) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:57.411593 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01189288-2268-405e-8676-365ec1a17194 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:57.413106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01189288-2268-405e-8676-365ec1a17194 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:57.421777 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 45/90] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:57:57 2022] GET /identity/v3/role_assignments?user.id=9c5fead1d17444a69ba60c5cf671f4c0&scope.project.id=b4cfdc9ec6a74df880eb9dcd5e97b98c&role.id=3e3756d9482f4b9b86271eacdc19f523 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:57.422643 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01189288-2268-405e-8676-365ec1a17194 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:57.459503 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b03fa-ba94-4241-b1b4-12993606b950 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:57.507121 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6202331b-0fc3-4518-9f6c-336a844b77b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:57.508030 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202331b-0fc3-4518-9f6c-336a844b77b7 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:57.508406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202331b-0fc3-4518-9f6c-336a844b77b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:57.508806 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202331b-0fc3-4518-9f6c-336a844b77b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:57.510116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6202331b-0fc3-4518-9f6c-336a844b77b7 None admin] RBAC: Adding query filter params (role.id=c426875a15834aa5b122e2a62267d15a, scope.project.id=b4cfdc9ec6a74df880eb9dcd5e97b98c, user.id=e6c6442222c54f34a84fdb4577045a9f) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:57.510681 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6202331b-0fc3-4518-9f6c-336a844b77b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:57.512677 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6202331b-0fc3-4518-9f6c-336a844b77b7 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:57.529173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 46/91] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:57:57 2022] GET /identity/v3/role_assignments?user.id=e6c6442222c54f34a84fdb4577045a9f&scope.project.id=b4cfdc9ec6a74df880eb9dcd5e97b98c&role.id=c426875a15834aa5b122e2a62267d15a => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:57.529173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6202331b-0fc3-4518-9f6c-336a844b77b7 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:57.579920 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:57.603293 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01189288-2268-405e-8676-365ec1a17194 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:57.665536 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ba018-09bf-4698-a7ba-8de03506a49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:57.666254 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ba018-09bf-4698-a7ba-8de03506a49c None admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:57.666628 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ba018-09bf-4698-a7ba-8de03506a49c None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:57.666970 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ba018-09bf-4698-a7ba-8de03506a49c None admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:57.667797 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117ba018-09bf-4698-a7ba-8de03506a49c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:57.670479 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117ba018-09bf-4698-a7ba-8de03506a49c None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:57.698534 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6202331b-0fc3-4518-9f6c-336a844b77b7 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:57.709682 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 46/92] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 14:57:57 2022] POST /identity/v3/users => generated 289 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:57:57.710925 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ba018-09bf-4698-a7ba-8de03506a49c None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:57.761665 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:57.810555 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb727f-292a-45b2-b2e3-bac939a57e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:57.811617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb727f-292a-45b2-b2e3-bac939a57e4e None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:57.812057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb727f-292a-45b2-b2e3-bac939a57e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:57.812674 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb727f-292a-45b2-b2e3-bac939a57e4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:57.813746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb727f-292a-45b2-b2e3-bac939a57e4e None admin] RBAC: Adding query filter params (role.id=b78105372b434d86872dd04e63dce750, scope.project.id=b4cfdc9ec6a74df880eb9dcd5e97b98c, user.id=9c5fead1d17444a69ba60c5cf671f4c0) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:57.814248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb727f-292a-45b2-b2e3-bac939a57e4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:57.816792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb727f-292a-45b2-b2e3-bac939a57e4e None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:57.837698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 47/93] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:57:57 2022] GET /identity/v3/role_assignments?user.id=9c5fead1d17444a69ba60c5cf671f4c0&scope.project.id=b4cfdc9ec6a74df880eb9dcd5e97b98c&role.id=b78105372b434d86872dd04e63dce750 => generated 259 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:57.867043 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ba018-09bf-4698-a7ba-8de03506a49c None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:57.930917 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05c454-9e04-434c-8037-e430fc5f9e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:57.931903 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05c454-9e04-434c-8037-e430fc5f9e92 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:57.932369 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05c454-9e04-434c-8037-e430fc5f9e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:57.932803 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05c454-9e04-434c-8037-e430fc5f9e92 None admin] PATH_INFO: `/v3/users/9c5fead1d17444a69ba60c5cf671f4c0` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:57.935146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05c454-9e04-434c-8037-e430fc5f9e92 None admin] RBAC: Authorizing `identity:get_user(user_id=9c5fead1d17444a69ba60c5cf671f4c0)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:57.937880 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05c454-9e04-434c-8037-e430fc5f9e92 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:57.941165 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 47/94] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:57:57 2022] GET /identity/v3/users/9c5fead1d17444a69ba60c5cf671f4c0 => generated 275 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:57.947464 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb727f-292a-45b2-b2e3-bac939a57e4e None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:58.158840 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:58.165546 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb727f-292a-45b2-b2e3-bac939a57e4e None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:58.220164 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf0ee8-02e2-4c6e-b91d-e86f87abdbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:58.220791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf0ee8-02e2-4c6e-b91d-e86f87abdbaf None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:58.221083 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf0ee8-02e2-4c6e-b91d-e86f87abdbaf None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:58.221397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf0ee8-02e2-4c6e-b91d-e86f87abdbaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:58.346156 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bf0ee8-02e2-4c6e-b91d-e86f87abdbaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:58.349299 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bf0ee8-02e2-4c6e-b91d-e86f87abdbaf None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:58.550182 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:58.561418 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 48/95] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:57:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:57:58.563866 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05c454-9e04-434c-8037-e430fc5f9e92 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:58.706617 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:57:58.712236 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05c454-9e04-434c-8037-e430fc5f9e92 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:58.783721 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235b6bb-88ec-4caa-a2e8-c7684771c224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:58.784864 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235b6bb-88ec-4caa-a2e8-c7684771c224 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:58.785359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235b6bb-88ec-4caa-a2e8-c7684771c224 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:58.785969 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235b6bb-88ec-4caa-a2e8-c7684771c224 None admin] PATH_INFO: `/v3/projects/2a392c80f2174a0083e4985ef028b967` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:58.788312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235b6bb-88ec-4caa-a2e8-c7684771c224 None admin] RBAC: Authorizing `identity:get_project(project_id=2a392c80f2174a0083e4985ef028b967)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:58.791083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235b6bb-88ec-4caa-a2e8-c7684771c224 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:58.794345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 48/96] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:57:58 2022] GET /identity/v3/projects/2a392c80f2174a0083e4985ef028b967 => generated 312 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:57:58.800826 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf0ee8-02e2-4c6e-b91d-e86f87abdbaf None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:57:58.924879 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf0ee8-02e2-4c6e-b91d-e86f87abdbaf None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:57:58.970592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3405358e-7803-485f-91b3-8f137f61fe13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:57:58.971288 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405358e-7803-485f-91b3-8f137f61fe13 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:57:58.971541 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405358e-7803-485f-91b3-8f137f61fe13 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:57:58.971811 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405358e-7803-485f-91b3-8f137f61fe13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:57:58.973030 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3405358e-7803-485f-91b3-8f137f61fe13 None admin] RBAC: Adding query filter params (role.id=3e3756d9482f4b9b86271eacdc19f523, scope.project.id=2a392c80f2174a0083e4985ef028b967, user.id=9c5fead1d17444a69ba60c5cf671f4c0) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:57:58.973396 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3405358e-7803-485f-91b3-8f137f61fe13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:57:58.974935 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3405358e-7803-485f-91b3-8f137f61fe13 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:57:58.984343 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 49/97] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:57:58 2022] GET /identity/v3/role_assignments?user.id=9c5fead1d17444a69ba60c5cf671f4c0&scope.project.id=2a392c80f2174a0083e4985ef028b967&role.id=3e3756d9482f4b9b86271eacdc19f523 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:00.414638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33071759-3b6b-4d26-be44-8e042173a1d4 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:00.415158 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33071759-3b6b-4d26-be44-8e042173a1d4 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:00.415656 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33071759-3b6b-4d26-be44-8e042173a1d4 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:00.418556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 49/98] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:00.433076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227b117-2f3e-4d3d-87db-22784c01819f None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:00.433620 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227b117-2f3e-4d3d-87db-22784c01819f None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:00.433938 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227b117-2f3e-4d3d-87db-22784c01819f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:00.469425 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-4227b117-2f3e-4d3d-87db-22784c01819f None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:00.471600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-4227b117-2f3e-4d3d-87db-22784c01819f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:00.553261 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:00.570004 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 50/99] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:01.066729 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64e8d3-f9d1-4a4d-a3aa-fbbb047cb318 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:01.067234 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64e8d3-f9d1-4a4d-a3aa-fbbb047cb318 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:01.067732 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64e8d3-f9d1-4a4d-a3aa-fbbb047cb318 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:01.070868 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 50/100] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:01.093875 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abd87e-3f46-4730-996a-6c96cdcb6de3 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:01.094494 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abd87e-3f46-4730-996a-6c96cdcb6de3 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:01.094976 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abd87e-3f46-4730-996a-6c96cdcb6de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:01.110700 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef64e8d3-f9d1-4a4d-a3aa-fbbb047cb318 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:01.133609 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-25abd87e-3f46-4730-996a-6c96cdcb6de3 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:01.136071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-25abd87e-3f46-4730-996a-6c96cdcb6de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:01.297450 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 51/101] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:58:01.302394 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef64e8d3-f9d1-4a4d-a3aa-fbbb047cb318 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:01.333386 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69edfc-dbf6-486a-9ea5-a77fc6f98a25 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:01.333968 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69edfc-dbf6-486a-9ea5-a77fc6f98a25 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:01.334441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69edfc-dbf6-486a-9ea5-a77fc6f98a25 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:01.336002 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 52/102] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:58:01.353212 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b644e3f-709c-46bc-a03c-eda85638929e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:01.354321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b644e3f-709c-46bc-a03c-eda85638929e None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:01.354796 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b644e3f-709c-46bc-a03c-eda85638929e None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:01.355253 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b644e3f-709c-46bc-a03c-eda85638929e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:01.359208 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a7420-2843-41ca-8d94-f082ee04d7b1 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:01.359719 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a7420-2843-41ca-8d94-f082ee04d7b1 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:01.360196 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a7420-2843-41ca-8d94-f082ee04d7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:01.387243 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b644e3f-709c-46bc-a03c-eda85638929e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:01.390853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-bc2a7420-2843-41ca-8d94-f082ee04d7b1 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:01.392261 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-bc2a7420-2843-41ca-8d94-f082ee04d7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:01.395503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-0b644e3f-709c-46bc-a03c-eda85638929e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:01.395503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:01.395503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:01.395503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:01.395503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:01.395503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:01.395503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:01.395503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:01.395503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:01.395503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:01.395503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:01.395503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:01.395503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:01.395503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:01.395503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:01.395503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:01.396366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:01.396366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:01.396366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:01.396366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:01.396366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:01.396366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:01.396366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:01.396366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:01.396366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:01.396366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:01.396366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:01.396366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:01.396366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:01.396366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:01.396366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:01.396366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:01.397114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:01.397114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:01.397114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:01.397114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:01.397114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:01.397114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:01.397114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:01.397114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:01.397114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:01.397114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:01.397114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:01.397114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:01.407352 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 51/103] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:58:01.417050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b644e3f-709c-46bc-a03c-eda85638929e None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:01.494484 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 53/104] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:58:01.500153 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:01.652402 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b644e3f-709c-46bc-a03c-eda85638929e None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:01.718066 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c758c1-a024-4c20-9962-3f6a58e9cc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:01.719133 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c758c1-a024-4c20-9962-3f6a58e9cc40 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:01.719634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c758c1-a024-4c20-9962-3f6a58e9cc40 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:01.720129 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c758c1-a024-4c20-9962-3f6a58e9cc40 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:01.721448 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c758c1-a024-4c20-9962-3f6a58e9cc40 None admin] RBAC: Adding query filter params (name=member) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:01.721984 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c758c1-a024-4c20-9962-3f6a58e9cc40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:01.724941 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c758c1-a024-4c20-9962-3f6a58e9cc40 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:01.741162 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 52/105] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 14:58:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:01.750176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a7420-2843-41ca-8d94-f082ee04d7b1 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:01.820036 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:01.833431 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c758c1-a024-4c20-9962-3f6a58e9cc40 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:01.882206 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a7420-2843-41ca-8d94-f082ee04d7b1 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:01.926409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44adeafc-b7f2-4799-a7a3-57df1789ad69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:01.927439 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44adeafc-b7f2-4799-a7a3-57df1789ad69 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:01.927903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44adeafc-b7f2-4799-a7a3-57df1789ad69 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:01.928381 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44adeafc-b7f2-4799-a7a3-57df1789ad69 None admin] PATH_INFO: `/v3/users/9c5fead1d17444a69ba60c5cf671f4c0` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:01.929745 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44adeafc-b7f2-4799-a7a3-57df1789ad69 None admin] RBAC: Authorizing `identity:get_user(user_id=9c5fead1d17444a69ba60c5cf671f4c0)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:01.931396 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44adeafc-b7f2-4799-a7a3-57df1789ad69 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:01.933324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 54/106] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:01 2022] GET /identity/v3/users/9c5fead1d17444a69ba60c5cf671f4c0 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:01.934446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44adeafc-b7f2-4799-a7a3-57df1789ad69 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:02.046904 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44adeafc-b7f2-4799-a7a3-57df1789ad69 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:02.051223 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:02.065044 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c758c1-a024-4c20-9962-3f6a58e9cc40 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:02.081280 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:02.098128 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f5d71-494e-4b1f-a7da-0c26a493b183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:02.099154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f5d71-494e-4b1f-a7da-0c26a493b183 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:02.099652 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f5d71-494e-4b1f-a7da-0c26a493b183 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:02.100106 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f5d71-494e-4b1f-a7da-0c26a493b183 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:02.112703 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67f5d71-494e-4b1f-a7da-0c26a493b183 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:02.118566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-a67f5d71-494e-4b1f-a7da-0c26a493b183 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:58:02.118566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:02.118566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:02.118566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:02.118566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:02.118566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:02.118566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:02.118566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:02.118566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:02.118566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:02.118566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:02.118566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:02.118566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:02.118566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:02.118566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:02.118566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:02.119507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:02.119507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:02.119507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:02.119507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:02.119507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:02.119507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:02.119507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:02.119507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:02.119507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:02.119507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:02.119507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:02.119507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:02.119507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:02.119507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:02.119507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:02.119507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:02.120306 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:02.120306 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:02.120306 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:02.120306 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:02.120306 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:02.120306 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:02.120306 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:02.120306 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:02.120306 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:02.120306 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:02.120306 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:58:02.120306 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:58:02.122756 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 55/107] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 14:58:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:02.124455 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aab973-cf34-4268-82fd-9571a4bbd0b0 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:02.124934 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aab973-cf34-4268-82fd-9571a4bbd0b0 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:02.125984 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aab973-cf34-4268-82fd-9571a4bbd0b0 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:02.127635 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 56/108] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:02.132896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aab973-cf34-4268-82fd-9571a4bbd0b0 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:02.138993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2bb5a-89b3-46ab-b919-f263a008d40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:02.139633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2bb5a-89b3-46ab-b919-f263a008d40e None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:02.140016 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2bb5a-89b3-46ab-b919-f263a008d40e None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:02.140436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2bb5a-89b3-46ab-b919-f263a008d40e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:02.154658 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2bb5a-89b3-46ab-b919-f263a008d40e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:02.160690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-cad2bb5a-89b3-46ab-b919-f263a008d40e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:58:02.160690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:02.160690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:02.160690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:02.160690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:02.160690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:02.160690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:02.160690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:02.160690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:02.160690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:02.160690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:02.160690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:02.160690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:02.160690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:02.160690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:02.160690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:02.161811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:02.161811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:02.161811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:02.161811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:02.161811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:02.161811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:02.161811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:02.161811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:02.161811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:02.161811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:02.161811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:02.161811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:02.161811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:02.161811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:02.161811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:02.161811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:02.162749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:02.162749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:02.162749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:02.162749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:02.162749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:02.162749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:02.162749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:02.162749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:02.162749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:02.162749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:02.162749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:58:02.162749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:02.164642 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 53/109] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 14:58:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:02.165480 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2bb5a-89b3-46ab-b919-f263a008d40e None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:02.257178 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aab973-cf34-4268-82fd-9571a4bbd0b0 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:02.282313 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2bb5a-89b3-46ab-b919-f263a008d40e None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:02.297759 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e678b83-3bbe-455f-8197-83dd0f4f70ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:02.298777 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e678b83-3bbe-455f-8197-83dd0f4f70ea None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:02.299158 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e678b83-3bbe-455f-8197-83dd0f4f70ea None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:02.299526 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e678b83-3bbe-455f-8197-83dd0f4f70ea None admin] PATH_INFO: `/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:02.300758 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e678b83-3bbe-455f-8197-83dd0f4f70ea None admin] RBAC: Authorizing `identity:get_project(project_id=b4cfdc9ec6a74df880eb9dcd5e97b98c)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:02.302367 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e678b83-3bbe-455f-8197-83dd0f4f70ea None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:02.309058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 57/110] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:02 2022] GET /identity/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:02.309058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b8221-b892-4327-9570-0885c50496dd None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:02.309058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b8221-b892-4327-9570-0885c50496dd None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:02.309058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b8221-b892-4327-9570-0885c50496dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:02.325634 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-698b8221-b892-4327-9570-0885c50496dd None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:02.326974 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-698b8221-b892-4327-9570-0885c50496dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:02.329408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef5851-1777-4198-b4a4-766627ff5b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:02.330057 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef5851-1777-4198-b4a4-766627ff5b73 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:02.330396 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef5851-1777-4198-b4a4-766627ff5b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:02.330674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef5851-1777-4198-b4a4-766627ff5b73 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:02.331455 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef5851-1777-4198-b4a4-766627ff5b73 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:02.331768 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef5851-1777-4198-b4a4-766627ff5b73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:02.333303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef5851-1777-4198-b4a4-766627ff5b73 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:02.343659 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 54/111] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:58:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:02.344613 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef5851-1777-4198-b4a4-766627ff5b73 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:02.363172 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:02.382621 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:02.426308 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 58/112] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:02.427971 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7b993-ac15-430e-9ec0-e0dc270ce2b9 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:02.428337 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7b993-ac15-430e-9ec0-e0dc270ce2b9 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:02.428709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7b993-ac15-430e-9ec0-e0dc270ce2b9 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:02.433199 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 59/113] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:02.433199 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc7b993-ac15-430e-9ec0-e0dc270ce2b9 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:02.503220 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef5851-1777-4198-b4a4-766627ff5b73 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:02.568753 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844db04a-8293-4e07-8f86-c219fbf9fc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:02.569982 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844db04a-8293-4e07-8f86-c219fbf9fc4c None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:02.570704 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844db04a-8293-4e07-8f86-c219fbf9fc4c None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:02.571282 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844db04a-8293-4e07-8f86-c219fbf9fc4c None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:02.572600 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844db04a-8293-4e07-8f86-c219fbf9fc4c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:02.573254 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844db04a-8293-4e07-8f86-c219fbf9fc4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:02.576588 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844db04a-8293-4e07-8f86-c219fbf9fc4c None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:02.593566 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 55/114] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:02.595309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844db04a-8293-4e07-8f86-c219fbf9fc4c None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:02.651707 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc7b993-ac15-430e-9ec0-e0dc270ce2b9 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:02.733392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b67e30-e478-421d-b1d6-b6428d754a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:02.734790 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b67e30-e478-421d-b1d6-b6428d754a98 None admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:02.735443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b67e30-e478-421d-b1d6-b6428d754a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:02.736078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b67e30-e478-421d-b1d6-b6428d754a98 None admin] PATH_INFO: `/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c/users/9c5fead1d17444a69ba60c5cf671f4c0/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:02.738133 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:02.741267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b67e30-e478-421d-b1d6-b6428d754a98 None admin] RBAC: Authorizing `identity:create_grant(project_id=b4cfdc9ec6a74df880eb9dcd5e97b98c, user_id=9c5fead1d17444a69ba60c5cf671f4c0, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:02.744813 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b67e30-e478-421d-b1d6-b6428d754a98 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:02.765468 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 60/115] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 14:58:02 2022] PUT /identity/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c/users/9c5fead1d17444a69ba60c5cf671f4c0/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:58:02.774947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bc93d-cf49-472a-9f4e-b46252875357 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:02.775317 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bc93d-cf49-472a-9f4e-b46252875357 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:02.775517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bc93d-cf49-472a-9f4e-b46252875357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:02.780841 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844db04a-8293-4e07-8f86-c219fbf9fc4c None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:02.804191 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-0b4bc93d-cf49-472a-9f4e-b46252875357 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:02.806590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-0b4bc93d-cf49-472a-9f4e-b46252875357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:02.832330 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72766bb-4b8f-4c19-a31f-d17b3d5b3aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:02.833317 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72766bb-4b8f-4c19-a31f-d17b3d5b3aca None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:02.833740 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72766bb-4b8f-4c19-a31f-d17b3d5b3aca None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:02.834150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72766bb-4b8f-4c19-a31f-d17b3d5b3aca None admin] PATH_INFO: `/v3/users/9c5fead1d17444a69ba60c5cf671f4c0` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:02.835741 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72766bb-4b8f-4c19-a31f-d17b3d5b3aca None admin] RBAC: Authorizing `identity:get_user(user_id=9c5fead1d17444a69ba60c5cf671f4c0)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:02.837266 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72766bb-4b8f-4c19-a31f-d17b3d5b3aca None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:02.846258 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 56/116] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:02 2022] GET /identity/v3/users/9c5fead1d17444a69ba60c5cf671f4c0 => generated 275 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:02.846258 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72766bb-4b8f-4c19-a31f-d17b3d5b3aca None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:02.860664 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:02.929757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 61/117] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:02.931196 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bc93d-cf49-472a-9f4e-b46252875357 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:03.015974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72766bb-4b8f-4c19-a31f-d17b3d5b3aca None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:03.074132 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3642c7-01d2-4ec9-a932-cfa04334b3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:03.075206 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3642c7-01d2-4ec9-a932-cfa04334b3d0 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:03.075523 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3642c7-01d2-4ec9-a932-cfa04334b3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:03.075822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3642c7-01d2-4ec9-a932-cfa04334b3d0 None admin] PATH_INFO: `/v3/users/e6c6442222c54f34a84fdb4577045a9f` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:03.078014 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3642c7-01d2-4ec9-a932-cfa04334b3d0 None admin] RBAC: Authorizing `identity:get_user(user_id=e6c6442222c54f34a84fdb4577045a9f)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:03.080450 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3642c7-01d2-4ec9-a932-cfa04334b3d0 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:03.082733 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 57/118] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:02 2022] GET /identity/v3/users/e6c6442222c54f34a84fdb4577045a9f => generated 247 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:03.085062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b9b0b-2143-4094-ab48-72f39842d440 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:03.085671 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b9b0b-2143-4094-ab48-72f39842d440 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:03.086273 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b9b0b-2143-4094-ab48-72f39842d440 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:03.088570 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 58/119] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:03.089614 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b9b0b-2143-4094-ab48-72f39842d440 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:03.106415 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:03.121792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bc93d-cf49-472a-9f4e-b46252875357 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:03.184241 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c4752-c3ab-493e-9a6f-665d5b0fc2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:03.185612 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c4752-c3ab-493e-9a6f-665d5b0fc2e1 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:03.185956 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c4752-c3ab-493e-9a6f-665d5b0fc2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:03.186468 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c4752-c3ab-493e-9a6f-665d5b0fc2e1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:03.202351 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c4752-c3ab-493e-9a6f-665d5b0fc2e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:03.211343 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-3f7c4752-c3ab-493e-9a6f-665d5b0fc2e1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:58:03.211343 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:03.211343 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:03.211343 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:03.211343 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:03.211343 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:03.211343 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:03.211343 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:03.211343 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:03.211343 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:03.211343 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:03.211343 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:03.211343 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:03.211343 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:03.211343 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:03.211343 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:03.212943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:03.212943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:03.212943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:03.212943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:03.212943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:03.212943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:03.212943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:03.212943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:03.212943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:03.212943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:03.212943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:03.212943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:03.212943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:03.212943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:03.212943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:03.212943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:03.222085 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:03.222085 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:03.222085 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:03.222085 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:03.222085 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:03.222085 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:03.222085 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:03.222085 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:03.222085 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:03.222085 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:03.222085 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:58:03.222085 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:58:03.222085 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 62/120] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:03.222085 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e91de-35dc-4ad0-abe4-cf28706f6805 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:03.223647 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e91de-35dc-4ad0-abe4-cf28706f6805 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:03.223647 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e91de-35dc-4ad0-abe4-cf28706f6805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:03.243115 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-402e91de-35dc-4ad0-abe4-cf28706f6805 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:03.244248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-402e91de-35dc-4ad0-abe4-cf28706f6805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:03.255450 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b9b0b-2143-4094-ab48-72f39842d440 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:03.269778 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:03.292513 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94489e92-5ffc-402e-8772-e8a53946ce92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:03.293236 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94489e92-5ffc-402e-8772-e8a53946ce92 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:03.293515 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94489e92-5ffc-402e-8772-e8a53946ce92 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:03.293895 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94489e92-5ffc-402e-8772-e8a53946ce92 None admin] PATH_INFO: `/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:03.295078 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94489e92-5ffc-402e-8772-e8a53946ce92 None admin] RBAC: Authorizing `identity:get_project(project_id=b4cfdc9ec6a74df880eb9dcd5e97b98c)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:03.296549 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94489e92-5ffc-402e-8772-e8a53946ce92 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:03.298525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 59/121] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:03 2022] GET /identity/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:03.299459 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94489e92-5ffc-402e-8772-e8a53946ce92 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:03.321146 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 63/122] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:03.326526 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e91de-35dc-4ad0-abe4-cf28706f6805 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:03.408407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:03.438026 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94489e92-5ffc-402e-8772-e8a53946ce92 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:03.479768 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faee818-d856-4ecc-bfb0-efdd646e30c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:03.481010 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faee818-d856-4ecc-bfb0-efdd646e30c3 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:03.481606 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faee818-d856-4ecc-bfb0-efdd646e30c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:03.482088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faee818-d856-4ecc-bfb0-efdd646e30c3 None admin] PATH_INFO: `/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:03.483671 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faee818-d856-4ecc-bfb0-efdd646e30c3 None admin] RBAC: Authorizing `identity:get_project(project_id=b4cfdc9ec6a74df880eb9dcd5e97b98c)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:03.485947 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faee818-d856-4ecc-bfb0-efdd646e30c3 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:03.488282 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 60/123] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:03 2022] GET /identity/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:03.489680 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faee818-d856-4ecc-bfb0-efdd646e30c3 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:03.517696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e91de-35dc-4ad0-abe4-cf28706f6805 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:03.581283 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f4c37-daf7-45b0-ae9e-d4b9d5052d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:03.582362 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f4c37-daf7-45b0-ae9e-d4b9d5052d41 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:03.583176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f4c37-daf7-45b0-ae9e-d4b9d5052d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:03.583472 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f4c37-daf7-45b0-ae9e-d4b9d5052d41 None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:03.584746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f4c37-daf7-45b0-ae9e-d4b9d5052d41 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:03.585677 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f4c37-daf7-45b0-ae9e-d4b9d5052d41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:03.588520 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f4c37-daf7-45b0-ae9e-d4b9d5052d41 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:03.610602 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 64/124] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 14:58:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:03.610602 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f4c37-daf7-45b0-ae9e-d4b9d5052d41 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:03.619223 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faee818-d856-4ecc-bfb0-efdd646e30c3 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:03.642108 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:03.669224 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd7685-a84e-403a-9efd-201a1d3757f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:03.670198 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd7685-a84e-403a-9efd-201a1d3757f9 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:03.670686 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd7685-a84e-403a-9efd-201a1d3757f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:03.671188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd7685-a84e-403a-9efd-201a1d3757f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:03.673056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd7685-a84e-403a-9efd-201a1d3757f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:03.675791 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd7685-a84e-403a-9efd-201a1d3757f9 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:03.678873 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 61/125] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:58:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:03.682297 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd7685-a84e-403a-9efd-201a1d3757f9 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:03.701876 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:03.729350 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f4c37-daf7-45b0-ae9e-d4b9d5052d41 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:03.767992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5051e-4e80-4789-b7b9-96c6ad73ad7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:03.768697 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5051e-4e80-4789-b7b9-96c6ad73ad7b None admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:03.768959 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5051e-4e80-4789-b7b9-96c6ad73ad7b None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:03.769263 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5051e-4e80-4789-b7b9-96c6ad73ad7b None admin] PATH_INFO: `/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c/users/9c5fead1d17444a69ba60c5cf671f4c0/roles/b78105372b434d86872dd04e63dce750` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:03.771560 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e5051e-4e80-4789-b7b9-96c6ad73ad7b None admin] RBAC: Authorizing `identity:create_grant(project_id=b4cfdc9ec6a74df880eb9dcd5e97b98c, user_id=9c5fead1d17444a69ba60c5cf671f4c0, role_id=b78105372b434d86872dd04e63dce750)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:03.773035 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e5051e-4e80-4789-b7b9-96c6ad73ad7b None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:03.787618 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 65/126] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 14:58:03 2022] PUT /identity/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c/users/9c5fead1d17444a69ba60c5cf671f4c0/roles/b78105372b434d86872dd04e63dce750 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:58:03.788902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5051e-4e80-4789-b7b9-96c6ad73ad7b None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:03.804208 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd7685-a84e-403a-9efd-201a1d3757f9 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:03.849443 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff2b89-2dab-4a1a-9b11-7aa95d9b21b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:03.850364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff2b89-2dab-4a1a-9b11-7aa95d9b21b6 None admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:03.850719 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff2b89-2dab-4a1a-9b11-7aa95d9b21b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:03.851072 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff2b89-2dab-4a1a-9b11-7aa95d9b21b6 None admin] PATH_INFO: `/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c/users/e6c6442222c54f34a84fdb4577045a9f/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:03.853578 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff2b89-2dab-4a1a-9b11-7aa95d9b21b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=b4cfdc9ec6a74df880eb9dcd5e97b98c, user_id=e6c6442222c54f34a84fdb4577045a9f, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:03.855425 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff2b89-2dab-4a1a-9b11-7aa95d9b21b6 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:03.873657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 62/127] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 14:58:03 2022] PUT /identity/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c/users/e6c6442222c54f34a84fdb4577045a9f/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:58:03.875013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff2b89-2dab-4a1a-9b11-7aa95d9b21b6 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:03.917335 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:03.973955 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5051e-4e80-4789-b7b9-96c6ad73ad7b None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:04.027503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:04.036944 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f3875-fd29-40cf-911e-8310132f5b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:04.038686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f3875-fd29-40cf-911e-8310132f5b14 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:04.039590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f3875-fd29-40cf-911e-8310132f5b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:04.040939 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f3875-fd29-40cf-911e-8310132f5b14 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:04.059184 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff2b89-2dab-4a1a-9b11-7aa95d9b21b6 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:04.069308 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f3875-fd29-40cf-911e-8310132f5b14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:04.080460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-5f1f3875-fd29-40cf-911e-8310132f5b14 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:04.080460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:04.080460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:04.080460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:04.080460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:04.080460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:04.080460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:04.080460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:04.080460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:04.080460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:04.080460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:04.080460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:04.080460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:04.080460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:04.080460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:04.080460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:04.082215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:04.082215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:04.082215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:04.082215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:04.082215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:04.082215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:04.082215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:04.082215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:04.082215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:04.082215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:04.082215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:04.082215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:04.082215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:04.082215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:04.082215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:04.082215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:04.083797 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:04.083797 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:04.083797 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:04.083797 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:04.083797 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:04.083797 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:04.083797 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:04.083797 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:04.083797 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:04.083797 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:04.083797 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:04.083797 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:58:04.089079 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 66/128] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:04.089079 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f3875-fd29-40cf-911e-8310132f5b14 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:04.135568 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf092a-fa78-4598-9871-74fa570d2447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:04.137915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf092a-fa78-4598-9871-74fa570d2447 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:04.138685 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf092a-fa78-4598-9871-74fa570d2447 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:04.140751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf092a-fa78-4598-9871-74fa570d2447 None admin] PATH_INFO: `/v3/users/f2a011a1ed924cb18c8a0c0f3bb6dcd4` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:04.161370 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf092a-fa78-4598-9871-74fa570d2447 None admin] RBAC: Authorizing `identity:get_user(user_id=f2a011a1ed924cb18c8a0c0f3bb6dcd4)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:04.162921 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf092a-fa78-4598-9871-74fa570d2447 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:04.166839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 63/129] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:03 2022] GET /identity/v3/users/f2a011a1ed924cb18c8a0c0f3bb6dcd4 => generated 289 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:04.166839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf092a-fa78-4598-9871-74fa570d2447 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:04.304477 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f3875-fd29-40cf-911e-8310132f5b14 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:04.347311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf092a-fa78-4598-9871-74fa570d2447 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:04.351531 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8febf2ac-cb34-4906-b972-bf2d8d9243d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:04.352186 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febf2ac-cb34-4906-b972-bf2d8d9243d7 None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:04.352469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febf2ac-cb34-4906-b972-bf2d8d9243d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:04.352740 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febf2ac-cb34-4906-b972-bf2d8d9243d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:04.353437 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8febf2ac-cb34-4906-b972-bf2d8d9243d7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:04.354915 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8febf2ac-cb34-4906-b972-bf2d8d9243d7 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:04.367262 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:04.395065 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 67/130] 10.222.0.44 () {60 vars in 1141 bytes} [Sun Sep 11 14:58:04 2022] POST /identity/v3/projects => generated 302 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:58:04.395801 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8febf2ac-cb34-4906-b972-bf2d8d9243d7 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:04.417534 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:04.428758 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3b21c-759b-4a99-9dfa-8f60157b0218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:04.429432 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3b21c-759b-4a99-9dfa-8f60157b0218 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:04.429714 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3b21c-759b-4a99-9dfa-8f60157b0218 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:04.429987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3b21c-759b-4a99-9dfa-8f60157b0218 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:04.430671 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3b21c-759b-4a99-9dfa-8f60157b0218 None admin] RBAC: Adding query filter params (name=member) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:04.430973 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3b21c-759b-4a99-9dfa-8f60157b0218 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:04.436218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3b21c-759b-4a99-9dfa-8f60157b0218 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:04.453635 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 64/131] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 14:58:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:04.459537 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3b21c-759b-4a99-9dfa-8f60157b0218 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:04.582631 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8febf2ac-cb34-4906-b972-bf2d8d9243d7 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:04.658664 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0fcbb-8c83-44a7-97ca-736851dc2d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:04.660725 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0fcbb-8c83-44a7-97ca-736851dc2d2a None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:04.662010 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0fcbb-8c83-44a7-97ca-736851dc2d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:04.662642 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0fcbb-8c83-44a7-97ca-736851dc2d2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:04.683464 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3b21c-759b-4a99-9dfa-8f60157b0218 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:04.796513 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:04.802885 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a7e5a-76a2-4e84-a1f5-9eb3420fc0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:04.804228 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a7e5a-76a2-4e84-a1f5-9eb3420fc0fa None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:04.804702 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a7e5a-76a2-4e84-a1f5-9eb3420fc0fa None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:04.805244 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a7e5a-76a2-4e84-a1f5-9eb3420fc0fa None admin] PATH_INFO: `/v3/users/9c5fead1d17444a69ba60c5cf671f4c0` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:04.810434 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a7e5a-76a2-4e84-a1f5-9eb3420fc0fa None admin] RBAC: Authorizing `identity:get_user(user_id=9c5fead1d17444a69ba60c5cf671f4c0)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:04.813192 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a7e5a-76a2-4e84-a1f5-9eb3420fc0fa None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:04.816061 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 65/132] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:04 2022] GET /identity/v3/users/9c5fead1d17444a69ba60c5cf671f4c0 => generated 275 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:04.824872 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0fcbb-8c83-44a7-97ca-736851dc2d2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:04.826805 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a7e5a-76a2-4e84-a1f5-9eb3420fc0fa None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:04.828862 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0fcbb-8c83-44a7-97ca-736851dc2d2a None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:05.019818 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:05.105178 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 68/133] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:58:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 701 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:58:05.105396 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a7e5a-76a2-4e84-a1f5-9eb3420fc0fa None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:05.106782 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0fcbb-8c83-44a7-97ca-736851dc2d2a None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:05.194389 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065524d3-265b-416d-89be-4dde83adbede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:05.195484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065524d3-265b-416d-89be-4dde83adbede None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:05.195974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065524d3-265b-416d-89be-4dde83adbede None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:05.196456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065524d3-265b-416d-89be-4dde83adbede None admin] PATH_INFO: `/v3/projects/2a392c80f2174a0083e4985ef028b967` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:05.202521 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065524d3-265b-416d-89be-4dde83adbede None admin] RBAC: Authorizing `identity:get_project(project_id=2a392c80f2174a0083e4985ef028b967)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:05.205273 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065524d3-265b-416d-89be-4dde83adbede None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:05.208477 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 66/134] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:04 2022] GET /identity/v3/projects/2a392c80f2174a0083e4985ef028b967 => generated 312 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:05.215025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065524d3-265b-416d-89be-4dde83adbede None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:05.248649 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:05.268556 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0fcbb-8c83-44a7-97ca-736851dc2d2a None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:05.339495 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3addd64-bc44-4d71-8d21-c19de01c142c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:05.341020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3addd64-bc44-4d71-8d21-c19de01c142c None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:05.341799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3addd64-bc44-4d71-8d21-c19de01c142c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:05.342518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3addd64-bc44-4d71-8d21-c19de01c142c None admin] PATH_INFO: `/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:05.344828 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3addd64-bc44-4d71-8d21-c19de01c142c None admin] RBAC: Authorizing `identity:get_project(project_id=b4cfdc9ec6a74df880eb9dcd5e97b98c)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:05.348294 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3addd64-bc44-4d71-8d21-c19de01c142c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:05.351756 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 69/135] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:05 2022] GET /identity/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:05.358809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3addd64-bc44-4d71-8d21-c19de01c142c None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:05.553040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065524d3-265b-416d-89be-4dde83adbede None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:05.607856 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:05.618607 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c001637-eb2c-4dd5-98f6-c4aed25f2a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:05.619802 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c001637-eb2c-4dd5-98f6-c4aed25f2a10 None admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:05.620326 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c001637-eb2c-4dd5-98f6-c4aed25f2a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:05.620831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c001637-eb2c-4dd5-98f6-c4aed25f2a10 None admin] PATH_INFO: `/v3/projects/2a392c80f2174a0083e4985ef028b967/users/9c5fead1d17444a69ba60c5cf671f4c0/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:05.623991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c001637-eb2c-4dd5-98f6-c4aed25f2a10 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a392c80f2174a0083e4985ef028b967, user_id=9c5fead1d17444a69ba60c5cf671f4c0, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:05.625792 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c001637-eb2c-4dd5-98f6-c4aed25f2a10 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:05.645211 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 67/136] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 14:58:05 2022] PUT /identity/v3/projects/2a392c80f2174a0083e4985ef028b967/users/9c5fead1d17444a69ba60c5cf671f4c0/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 14:58:05.648394 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc5279-6c1f-4772-ada5-2dd0ebcb20e5 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:05.649022 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc5279-6c1f-4772-ada5-2dd0ebcb20e5 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:05.649562 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc5279-6c1f-4772-ada5-2dd0ebcb20e5 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:05.651673 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 68/137] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:05.660796 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efb918-5aa7-47e7-9f18-1a8047cc6156 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:05.661482 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efb918-5aa7-47e7-9f18-1a8047cc6156 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:05.661693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3addd64-bc44-4d71-8d21-c19de01c142c None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:05.662154 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efb918-5aa7-47e7-9f18-1a8047cc6156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:05.707072 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-a1efb918-5aa7-47e7-9f18-1a8047cc6156 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:05.709950 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-a1efb918-5aa7-47e7-9f18-1a8047cc6156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:05.799646 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9bd3fd-bd43-4b12-932e-171fb4adad59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:05.800447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9bd3fd-bd43-4b12-932e-171fb4adad59 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:05.800569 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9bd3fd-bd43-4b12-932e-171fb4adad59 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:05.800934 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9bd3fd-bd43-4b12-932e-171fb4adad59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:05.801684 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9bd3fd-bd43-4b12-932e-171fb4adad59 None admin] RBAC: Adding query filter params (role.id=81d58cd096394c3aad885ad2d6f60f2c, scope.project.id=b4cfdc9ec6a74df880eb9dcd5e97b98c, user.id=f2a011a1ed924cb18c8a0c0f3bb6dcd4) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:05.801998 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9bd3fd-bd43-4b12-932e-171fb4adad59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:05.803707 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9bd3fd-bd43-4b12-932e-171fb4adad59 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:05.817294 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 70/138] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:58:05 2022] GET /identity/v3/role_assignments?user.id=f2a011a1ed924cb18c8a0c0f3bb6dcd4&scope.project.id=b4cfdc9ec6a74df880eb9dcd5e97b98c&role.id=81d58cd096394c3aad885ad2d6f60f2c => generated 259 bytes in 459 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:05.855598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 69/139] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 11 14:58:06.412033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9bd3fd-bd43-4b12-932e-171fb4adad59 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:06.452899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:06.575730 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9bd3fd-bd43-4b12-932e-171fb4adad59 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:06.623499 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60241e7-93e8-4b92-98c2-e2e92708105b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:06.624152 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60241e7-93e8-4b92-98c2-e2e92708105b None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:06.624430 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60241e7-93e8-4b92-98c2-e2e92708105b None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:06.624695 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60241e7-93e8-4b92-98c2-e2e92708105b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:06.647485 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60241e7-93e8-4b92-98c2-e2e92708105b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:06.653696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-e60241e7-93e8-4b92-98c2-e2e92708105b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:06.653696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:06.653696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:06.653696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:06.653696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:06.653696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:06.653696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:06.653696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:06.653696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:06.653696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:06.653696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:06.653696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:06.653696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:06.653696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:06.653696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:06.653696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:06.654606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:06.654606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:06.654606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:06.654606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:06.654606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:06.654606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:06.654606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:06.654606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:06.654606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:06.654606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:06.654606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:06.654606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:06.654606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:06.654606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:06.654606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:06.654606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:06.655357 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:06.655357 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:06.655357 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:06.655357 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:06.655357 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:06.655357 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:06.655357 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:06.655357 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:06.655357 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:06.655357 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:06.655357 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:06.655357 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:58:06.655357 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 71/140] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:06.662192 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efb918-5aa7-47e7-9f18-1a8047cc6156 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:06.806564 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efb918-5aa7-47e7-9f18-1a8047cc6156 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:06.829691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:06.852569 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221553da-d629-4c81-a410-41cd299e223a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:06.853621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221553da-d629-4c81-a410-41cd299e223a None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:06.854096 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221553da-d629-4c81-a410-41cd299e223a None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:06.854579 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221553da-d629-4c81-a410-41cd299e223a None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:06.855454 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221553da-d629-4c81-a410-41cd299e223a None admin] RBAC: Adding query filter params (name=member) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:06.855980 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221553da-d629-4c81-a410-41cd299e223a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:06.857747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221553da-d629-4c81-a410-41cd299e223a None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:06.867398 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 70/141] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 14:58:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:06.878297 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60241e7-93e8-4b92-98c2-e2e92708105b None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:07.018272 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:07.037558 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60241e7-93e8-4b92-98c2-e2e92708105b None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:07.084782 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43552775-12d9-40eb-9067-017255789b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:07.085378 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43552775-12d9-40eb-9067-017255789b0c None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:07.085650 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43552775-12d9-40eb-9067-017255789b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:07.085913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43552775-12d9-40eb-9067-017255789b0c None admin] PATH_INFO: `/v3/users/9c5fead1d17444a69ba60c5cf671f4c0` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:07.087094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43552775-12d9-40eb-9067-017255789b0c None admin] RBAC: Authorizing `identity:get_user(user_id=9c5fead1d17444a69ba60c5cf671f4c0)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:07.088563 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43552775-12d9-40eb-9067-017255789b0c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:07.090169 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 72/142] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:06 2022] GET /identity/v3/users/9c5fead1d17444a69ba60c5cf671f4c0 => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:07.096106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221553da-d629-4c81-a410-41cd299e223a None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:07.238290 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489065c-ba85-49b7-9ca0-616138d499b2 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:07.238725 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489065c-ba85-49b7-9ca0-616138d499b2 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:07.238842 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489065c-ba85-49b7-9ca0-616138d499b2 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:07.241811 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 73/143] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:58:07.249158 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb21f8-1496-44d2-b4a2-7ef819e1b97f None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:07.249158 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb21f8-1496-44d2-b4a2-7ef819e1b97f None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:07.249158 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb21f8-1496-44d2-b4a2-7ef819e1b97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:07.252578 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221553da-d629-4c81-a410-41cd299e223a None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:07.281264 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-29bb21f8-1496-44d2-b4a2-7ef819e1b97f None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:07.282943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-29bb21f8-1496-44d2-b4a2-7ef819e1b97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:07.304567 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b027dd70-085c-41b6-8db3-56430c189aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:07.311992 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027dd70-085c-41b6-8db3-56430c189aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:07.312588 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027dd70-085c-41b6-8db3-56430c189aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:07.313071 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027dd70-085c-41b6-8db3-56430c189aa2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:07.419935 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 74/144] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:07.422875 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:07.446955 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b027dd70-085c-41b6-8db3-56430c189aa2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:07.448743 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b027dd70-085c-41b6-8db3-56430c189aa2 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:07.576144 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 71/145] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:58:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 14:58:07.584785 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb21f8-1496-44d2-b4a2-7ef819e1b97f None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:07.702587 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:07.732113 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb21f8-1496-44d2-b4a2-7ef819e1b97f None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:07.775846 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e18c5-7a6b-4f2a-a81e-86421977e753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:07.776511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e18c5-7a6b-4f2a-a81e-86421977e753 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:07.776786 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e18c5-7a6b-4f2a-a81e-86421977e753 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:07.777054 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e18c5-7a6b-4f2a-a81e-86421977e753 None admin] PATH_INFO: `/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:07.778510 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64e18c5-7a6b-4f2a-a81e-86421977e753 None admin] RBAC: Authorizing `identity:get_project(project_id=b4cfdc9ec6a74df880eb9dcd5e97b98c)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:07.780079 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64e18c5-7a6b-4f2a-a81e-86421977e753 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:07.782277 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 75/146] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:07 2022] GET /identity/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:07.788651 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b027dd70-085c-41b6-8db3-56430c189aa2 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:07.895033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:07.916515 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b027dd70-085c-41b6-8db3-56430c189aa2 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:07.930030 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e18c5-7a6b-4f2a-a81e-86421977e753 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:07.968926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171de90-5568-4f1b-9b4c-4c7559ed25df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:07.970233 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171de90-5568-4f1b-9b4c-4c7559ed25df None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:07.970714 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171de90-5568-4f1b-9b4c-4c7559ed25df None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:07.971294 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171de90-5568-4f1b-9b4c-4c7559ed25df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:07.972293 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171de90-5568-4f1b-9b4c-4c7559ed25df None admin] RBAC: Adding query filter params (role.id=3e3756d9482f4b9b86271eacdc19f523, scope.project.id=b4cfdc9ec6a74df880eb9dcd5e97b98c, user.id=9c5fead1d17444a69ba60c5cf671f4c0) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:07.973275 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171de90-5568-4f1b-9b4c-4c7559ed25df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:07.975104 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171de90-5568-4f1b-9b4c-4c7559ed25df None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:07.989326 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 72/147] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:58:07 2022] GET /identity/v3/role_assignments?user.id=9c5fead1d17444a69ba60c5cf671f4c0&scope.project.id=b4cfdc9ec6a74df880eb9dcd5e97b98c&role.id=3e3756d9482f4b9b86271eacdc19f523 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:08.018989 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acbc8d-79f3-400b-980f-e97fe7e51791 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:08.019730 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acbc8d-79f3-400b-980f-e97fe7e51791 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:08.020200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acbc8d-79f3-400b-980f-e97fe7e51791 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:08.021942 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 73/148] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:58:08.030151 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1652f0-6eef-4372-8e12-48df113b0600 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:08.031477 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1652f0-6eef-4372-8e12-48df113b0600 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:08.032025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1652f0-6eef-4372-8e12-48df113b0600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:08.065233 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-bf1652f0-6eef-4372-8e12-48df113b0600 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:08.067211 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-bf1652f0-6eef-4372-8e12-48df113b0600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:08.141197 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e18c5-7a6b-4f2a-a81e-86421977e753 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:08.171221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 74/149] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:58:08.210085 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98debb4-36c1-4f8d-977f-75a318761f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:08.210766 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98debb4-36c1-4f8d-977f-75a318761f93 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:08.211039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98debb4-36c1-4f8d-977f-75a318761f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:08.211307 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98debb4-36c1-4f8d-977f-75a318761f93 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:08.228073 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98debb4-36c1-4f8d-977f-75a318761f93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:08.233057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-c98debb4-36c1-4f8d-977f-75a318761f93 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:58:08.233057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:08.233057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:08.233057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:08.233057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:08.233057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:08.233057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:08.233057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:08.233057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:08.233057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:08.233057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:08.233057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:08.233057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:08.233057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:08.233057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:08.233057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:08.233990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:08.233990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:08.233990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:08.233990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:08.233990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:08.233990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:08.233990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:08.233990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:08.233990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:08.233990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:08.233990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:08.233990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:08.233990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:08.233990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:08.233990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:08.233990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:08.234848 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:08.234848 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:08.234848 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:08.234848 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:08.234848 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:08.234848 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:08.234848 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:08.234848 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:08.234848 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:08.234848 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:08.234848 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:58:08.234848 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:58:08.235991 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 76/150] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 14:58:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:08.242861 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1652f0-6eef-4372-8e12-48df113b0600 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:08.341908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:08.379514 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1652f0-6eef-4372-8e12-48df113b0600 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:08.425889 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c72b85-0eed-4bae-aa48-ba7d8647f8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:08.426852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c72b85-0eed-4bae-aa48-ba7d8647f8c5 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:08.427170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c72b85-0eed-4bae-aa48-ba7d8647f8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:08.427464 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c72b85-0eed-4bae-aa48-ba7d8647f8c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:08.428404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c72b85-0eed-4bae-aa48-ba7d8647f8c5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:08.428707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c72b85-0eed-4bae-aa48-ba7d8647f8c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:08.430709 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c72b85-0eed-4bae-aa48-ba7d8647f8c5 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:08.442635 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 75/151] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:58:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:08.450970 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98debb4-36c1-4f8d-977f-75a318761f93 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:08.467366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:08.581352 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98debb4-36c1-4f8d-977f-75a318761f93 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:08.622525 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d95c1-2948-4fa3-aa0f-9ed3600d81ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:08.623186 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d95c1-2948-4fa3-aa0f-9ed3600d81ee None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:08.623509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d95c1-2948-4fa3-aa0f-9ed3600d81ee None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:08.623783 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d95c1-2948-4fa3-aa0f-9ed3600d81ee None admin] PATH_INFO: `/v3/users/9c5fead1d17444a69ba60c5cf671f4c0` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:08.625063 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d95c1-2948-4fa3-aa0f-9ed3600d81ee None admin] RBAC: Authorizing `identity:get_user(user_id=9c5fead1d17444a69ba60c5cf671f4c0)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:08.626574 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d95c1-2948-4fa3-aa0f-9ed3600d81ee None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:08.628418 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 77/152] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:08 2022] GET /identity/v3/users/9c5fead1d17444a69ba60c5cf671f4c0 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:08.634049 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c72b85-0eed-4bae-aa48-ba7d8647f8c5 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:08.751520 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c72b85-0eed-4bae-aa48-ba7d8647f8c5 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:08.793925 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb7a36-073b-4a45-9425-a7c5bd70bec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:08.794565 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb7a36-073b-4a45-9425-a7c5bd70bec0 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:08.794842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb7a36-073b-4a45-9425-a7c5bd70bec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:08.795108 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb7a36-073b-4a45-9425-a7c5bd70bec0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:08.804535 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:08.822491 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d95c1-2948-4fa3-aa0f-9ed3600d81ee None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:08.911633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bb7a36-073b-4a45-9425-a7c5bd70bec0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:08.913059 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bb7a36-073b-4a45-9425-a7c5bd70bec0 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:09.008200 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:09.040276 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d95c1-2948-4fa3-aa0f-9ed3600d81ee None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:09.048372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 76/153] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:58:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:58:09.049528 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4be83b-17a9-444d-91af-b03c331bb52b None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:09.049945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4be83b-17a9-444d-91af-b03c331bb52b None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:09.050515 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4be83b-17a9-444d-91af-b03c331bb52b None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:09.052712 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 77/154] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:09.058410 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4be83b-17a9-444d-91af-b03c331bb52b None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:09.103196 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675d6b7-94f3-4113-b4d5-4c3df210572e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:09.103853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675d6b7-94f3-4113-b4d5-4c3df210572e None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:09.104131 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675d6b7-94f3-4113-b4d5-4c3df210572e None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:09.104502 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675d6b7-94f3-4113-b4d5-4c3df210572e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:09.115793 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:09.120366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5675d6b7-94f3-4113-b4d5-4c3df210572e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:09.126094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-5675d6b7-94f3-4113-b4d5-4c3df210572e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:58:09.126094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:09.126094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:09.126094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:09.126094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:09.126094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:09.126094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:09.126094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:09.126094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:09.126094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:09.126094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:09.126094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:09.126094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:09.126094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:09.126094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:09.126094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:09.127167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:09.127167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:09.127167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:09.127167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:09.127167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:09.127167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:09.127167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:09.127167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:09.127167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:09.127167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:09.127167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:09.127167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:09.127167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:09.127167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:09.127167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:09.127167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:09.128194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:09.128194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:09.128194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:09.128194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:09.128194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:09.128194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:09.128194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:09.128194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:09.128194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:09.128194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:09.128194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:58:09.128194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:58:09.128194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 78/155] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 14:58:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:09.129756 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bceb62d-0a42-4092-95d5-8b31d4368480 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:09.132945 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bceb62d-0a42-4092-95d5-8b31d4368480 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:09.132945 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bceb62d-0a42-4092-95d5-8b31d4368480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:09.154585 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-1bceb62d-0a42-4092-95d5-8b31d4368480 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:09.155981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-1bceb62d-0a42-4092-95d5-8b31d4368480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:09.213536 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4be83b-17a9-444d-91af-b03c331bb52b None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:09.251447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 79/156] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:09.254606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bceb62d-0a42-4092-95d5-8b31d4368480 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:09.325932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527dfbb-caec-4a95-b42c-7c65ee304b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:09.327004 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527dfbb-caec-4a95-b42c-7c65ee304b9d None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:09.327471 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527dfbb-caec-4a95-b42c-7c65ee304b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:09.327926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527dfbb-caec-4a95-b42c-7c65ee304b9d None admin] PATH_INFO: `/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:09.329335 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6527dfbb-caec-4a95-b42c-7c65ee304b9d None admin] RBAC: Authorizing `identity:get_project(project_id=b4cfdc9ec6a74df880eb9dcd5e97b98c)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:09.330997 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6527dfbb-caec-4a95-b42c-7c65ee304b9d None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:09.333200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 78/157] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:09 2022] GET /identity/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:09.334582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834fe95-973b-494f-b630-f8fed7a95320 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:09.335074 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834fe95-973b-494f-b630-f8fed7a95320 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:09.335530 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9834fe95-973b-494f-b630-f8fed7a95320 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:09.336962 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 79/158] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:09.343343 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834fe95-973b-494f-b630-f8fed7a95320 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:09.393818 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:09.443968 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bceb62d-0a42-4092-95d5-8b31d4368480 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:09.483435 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfdbb4-204e-4dcd-9fbd-e2da9c893b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:09.484488 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfdbb4-204e-4dcd-9fbd-e2da9c893b65 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:09.484989 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfdbb4-204e-4dcd-9fbd-e2da9c893b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:09.485416 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfdbb4-204e-4dcd-9fbd-e2da9c893b65 None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:09.486606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcfdbb4-204e-4dcd-9fbd-e2da9c893b65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:09.487130 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcfdbb4-204e-4dcd-9fbd-e2da9c893b65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:09.488829 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcfdbb4-204e-4dcd-9fbd-e2da9c893b65 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:09.502308 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 80/159] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:09.503554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af95c0-7e52-4241-af6e-abd194fa5c9c None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:09.503831 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af95c0-7e52-4241-af6e-abd194fa5c9c None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:09.504093 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af95c0-7e52-4241-af6e-abd194fa5c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:09.517512 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9834fe95-973b-494f-b630-f8fed7a95320 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:09.525415 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:09.527799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-f5af95c0-7e52-4241-af6e-abd194fa5c9c None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:09.529506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f5af95c0-7e52-4241-af6e-abd194fa5c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:09.555908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e66dbf-34e6-43aa-8034-5d9247bc78c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:09.556779 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e66dbf-34e6-43aa-8034-5d9247bc78c5 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:09.557056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e66dbf-34e6-43aa-8034-5d9247bc78c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:09.557357 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e66dbf-34e6-43aa-8034-5d9247bc78c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:09.558094 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e66dbf-34e6-43aa-8034-5d9247bc78c5 None admin] RBAC: Adding query filter params (role.id=b78105372b434d86872dd04e63dce750, scope.project.id=b4cfdc9ec6a74df880eb9dcd5e97b98c, user.id=9c5fead1d17444a69ba60c5cf671f4c0) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:09.558408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e66dbf-34e6-43aa-8034-5d9247bc78c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:09.559898 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e66dbf-34e6-43aa-8034-5d9247bc78c5 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:09.565826 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 80/160] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:58:09 2022] GET /identity/v3/role_assignments?user.id=9c5fead1d17444a69ba60c5cf671f4c0&scope.project.id=b4cfdc9ec6a74df880eb9dcd5e97b98c&role.id=b78105372b434d86872dd04e63dce750 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:09.567170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb6f94-1443-4d9c-a304-5bf45ae760ba None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:09.567627 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb6f94-1443-4d9c-a304-5bf45ae760ba None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:09.567895 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb6f94-1443-4d9c-a304-5bf45ae760ba None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:09.569354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 81/161] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:09.569822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb6f94-1443-4d9c-a304-5bf45ae760ba None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:09.621396 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 81/162] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:09.621396 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af95c0-7e52-4241-af6e-abd194fa5c9c None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:09.742542 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:09.760042 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb6f94-1443-4d9c-a304-5bf45ae760ba None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:09.814803 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af95c0-7e52-4241-af6e-abd194fa5c9c None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:09.823226 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be4745-deb7-461b-976f-61e064868744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:09.824424 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be4745-deb7-461b-976f-61e064868744 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:09.824424 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be4745-deb7-461b-976f-61e064868744 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:09.824599 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be4745-deb7-461b-976f-61e064868744 None admin] PATH_INFO: `/v3/users/e6c6442222c54f34a84fdb4577045a9f` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:09.826656 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be4745-deb7-461b-976f-61e064868744 None admin] RBAC: Authorizing `identity:get_user(user_id=e6c6442222c54f34a84fdb4577045a9f)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:09.829107 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be4745-deb7-461b-976f-61e064868744 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:09.829941 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 82/163] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:09 2022] GET /identity/v3/users/e6c6442222c54f34a84fdb4577045a9f => generated 247 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:09.830997 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffdd99-715a-485a-b6db-a0bbf153b7b6 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:09.831339 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffdd99-715a-485a-b6db-a0bbf153b7b6 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:09.831627 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffdd99-715a-485a-b6db-a0bbf153b7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:09.851968 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-20ffdd99-715a-485a-b6db-a0bbf153b7b6 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:09.854101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-20ffdd99-715a-485a-b6db-a0bbf153b7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:09.871080 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264b2ad-37db-433e-a533-a8505ce326df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:09.871760 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264b2ad-37db-433e-a533-a8505ce326df None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:09.872076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264b2ad-37db-433e-a533-a8505ce326df None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:09.872353 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264b2ad-37db-433e-a533-a8505ce326df None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:09.885736 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264b2ad-37db-433e-a533-a8505ce326df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:09.892397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-e264b2ad-37db-433e-a533-a8505ce326df None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:58:09.892397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:09.892397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:09.892397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:09.892397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:09.892397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:09.892397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:09.892397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:09.892397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:09.892397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:09.892397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:09.892397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:09.892397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:09.892397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:09.892397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:09.892397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:09.893370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:09.893370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:09.893370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:09.893370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:09.893370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:09.893370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:09.893370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:09.893370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:09.893370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:09.893370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:09.893370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:09.893370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:09.893370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:09.893370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:09.893370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:09.893370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:09.894166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:09.894166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:09.894166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:09.894166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:09.894166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:09.894166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:09.894166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:09.894166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:09.894166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:09.894166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:09.894166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:58:09.894166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:58:09.894997 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 82/164] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:09.896060 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:09.899203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264b2ad-37db-433e-a533-a8505ce326df None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:09.935701 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 83/165] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:09.936541 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffdd99-715a-485a-b6db-a0bbf153b7b6 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:10.009190 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264b2ad-37db-433e-a533-a8505ce326df None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:10.029834 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:10.046883 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38210d-939f-4887-adb8-e8f39f03694a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:10.047501 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38210d-939f-4887-adb8-e8f39f03694a None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:10.047777 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38210d-939f-4887-adb8-e8f39f03694a None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:10.048045 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38210d-939f-4887-adb8-e8f39f03694a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:10.048390 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffdd99-715a-485a-b6db-a0bbf153b7b6 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:10.086486 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6814ff-34f6-4d30-971b-a536d3ce84eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:10.087134 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6814ff-34f6-4d30-971b-a536d3ce84eb None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:10.087404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6814ff-34f6-4d30-971b-a536d3ce84eb None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:10.087676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6814ff-34f6-4d30-971b-a536d3ce84eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:10.089052 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6814ff-34f6-4d30-971b-a536d3ce84eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:10.090404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6814ff-34f6-4d30-971b-a536d3ce84eb None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:10.092091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 84/166] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:58:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:10.096068 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6814ff-34f6-4d30-971b-a536d3ce84eb None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:10.134598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:10.136199 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38210d-939f-4887-adb8-e8f39f03694a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:10.138586 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38210d-939f-4887-adb8-e8f39f03694a None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:10.220278 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6814ff-34f6-4d30-971b-a536d3ce84eb None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:10.250202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 83/167] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:58:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:58:10.250752 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38210d-939f-4887-adb8-e8f39f03694a None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:10.264182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdfa34-06a2-442a-bc16-06c3ce02c1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:10.264800 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdfa34-06a2-442a-bc16-06c3ce02c1dd None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:10.265075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdfa34-06a2-442a-bc16-06c3ce02c1dd None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:10.265364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdfa34-06a2-442a-bc16-06c3ce02c1dd None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:10.266025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfdfa34-06a2-442a-bc16-06c3ce02c1dd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:10.266315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfdfa34-06a2-442a-bc16-06c3ce02c1dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:10.267695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfdfa34-06a2-442a-bc16-06c3ce02c1dd None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:10.272680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:10.276879 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 85/168] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 14:58:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:10.277719 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdfa34-06a2-442a-bc16-06c3ce02c1dd None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:10.353225 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38210d-939f-4887-adb8-e8f39f03694a None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:10.372133 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:10.375805 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdfa34-06a2-442a-bc16-06c3ce02c1dd None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:10.386350 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fea97a-a736-445d-9dfb-2f0fb44dc165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:10.386957 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fea97a-a736-445d-9dfb-2f0fb44dc165 None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:10.387233 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fea97a-a736-445d-9dfb-2f0fb44dc165 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:10.387567 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fea97a-a736-445d-9dfb-2f0fb44dc165 None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:10.388214 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fea97a-a736-445d-9dfb-2f0fb44dc165 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:10.389706 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fea97a-a736-445d-9dfb-2f0fb44dc165 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:10.410163 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82c0e5-9712-4cd9-b24c-83258bbfe19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:10.410798 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82c0e5-9712-4cd9-b24c-83258bbfe19f None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:10.416977 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 84/169] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 14:58:10 2022] POST /identity/v3/users => generated 283 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:58:10.417091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82c0e5-9712-4cd9-b24c-83258bbfe19f None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:10.417091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82c0e5-9712-4cd9-b24c-83258bbfe19f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:10.417253 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fea97a-a736-445d-9dfb-2f0fb44dc165 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:10.436367 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a82c0e5-9712-4cd9-b24c-83258bbfe19f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:10.441930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-7a82c0e5-9712-4cd9-b24c-83258bbfe19f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:10.441930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:10.441930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:10.441930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:10.441930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:10.441930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:10.441930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:10.441930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:10.441930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:10.441930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:10.441930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:10.441930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:10.441930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:10.441930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:10.441930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:10.441930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:10.443112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:10.443112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:10.443112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:10.443112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:10.443112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:10.443112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:10.443112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:10.443112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:10.443112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:10.443112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:10.443112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:10.443112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:10.443112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:10.443112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:10.443112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:10.443112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:10.444043 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:10.444043 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:10.444043 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:10.444043 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:10.444043 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:10.444043 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:10.444043 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:10.444043 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:10.444043 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:10.444043 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:10.444043 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:10.444043 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:10.444043 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 86/170] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:10.449505 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82c0e5-9712-4cd9-b24c-83258bbfe19f None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:10.527415 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:10.555732 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fea97a-a736-445d-9dfb-2f0fb44dc165 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:10.585353 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82c0e5-9712-4cd9-b24c-83258bbfe19f None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:10.600112 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd80a7-e9cc-4857-9c85-840f21b78969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:10.600771 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd80a7-e9cc-4857-9c85-840f21b78969 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:10.601047 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd80a7-e9cc-4857-9c85-840f21b78969 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:10.601345 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd80a7-e9cc-4857-9c85-840f21b78969 None admin] PATH_INFO: `/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:10.602532 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd80a7-e9cc-4857-9c85-840f21b78969 None admin] RBAC: Authorizing `identity:get_project(project_id=b4cfdc9ec6a74df880eb9dcd5e97b98c)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:10.604006 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd80a7-e9cc-4857-9c85-840f21b78969 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:10.605870 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 85/171] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:10 2022] GET /identity/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:10.611780 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd80a7-e9cc-4857-9c85-840f21b78969 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:10.628947 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059aeb8-a33d-4843-8e10-9a7f45c1ed3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:10.629873 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059aeb8-a33d-4843-8e10-9a7f45c1ed3e None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:10.630276 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059aeb8-a33d-4843-8e10-9a7f45c1ed3e None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:10.630663 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059aeb8-a33d-4843-8e10-9a7f45c1ed3e None admin] PATH_INFO: `/v3/users/f2a011a1ed924cb18c8a0c0f3bb6dcd4` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:10.631960 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059aeb8-a33d-4843-8e10-9a7f45c1ed3e None admin] RBAC: Authorizing `identity:get_user(user_id=f2a011a1ed924cb18c8a0c0f3bb6dcd4)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:10.633633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059aeb8-a33d-4843-8e10-9a7f45c1ed3e None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:10.635236 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 87/172] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:10 2022] GET /identity/v3/users/f2a011a1ed924cb18c8a0c0f3bb6dcd4 => generated 289 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:10.636040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059aeb8-a33d-4843-8e10-9a7f45c1ed3e None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:10.669662 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:10.739510 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd80a7-e9cc-4857-9c85-840f21b78969 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:10.766854 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059aeb8-a33d-4843-8e10-9a7f45c1ed3e None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:10.782278 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d9caf-7cc0-4759-b799-73f00fe1ba89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:10.783203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d9caf-7cc0-4759-b799-73f00fe1ba89 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:10.783479 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d9caf-7cc0-4759-b799-73f00fe1ba89 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:10.783753 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d9caf-7cc0-4759-b799-73f00fe1ba89 None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:10.784431 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d9caf-7cc0-4759-b799-73f00fe1ba89 None admin] RBAC: Adding query filter params (name=member) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:10.784764 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d9caf-7cc0-4759-b799-73f00fe1ba89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:10.786342 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d9caf-7cc0-4759-b799-73f00fe1ba89 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:10.795581 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 86/173] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 14:58:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:10.798448 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d9caf-7cc0-4759-b799-73f00fe1ba89 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:10.807582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95f3c3-5f0c-41bc-9bbc-af4040b15ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:10.808213 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95f3c3-5f0c-41bc-9bbc-af4040b15ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:10.808476 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95f3c3-5f0c-41bc-9bbc-af4040b15ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:10.808743 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95f3c3-5f0c-41bc-9bbc-af4040b15ab8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:10.811466 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f95f3c3-5f0c-41bc-9bbc-af4040b15ab8 None admin] RBAC: Adding query filter params (role.id=c426875a15834aa5b122e2a62267d15a, scope.project.id=b4cfdc9ec6a74df880eb9dcd5e97b98c, user.id=e6c6442222c54f34a84fdb4577045a9f) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:10.811466 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f95f3c3-5f0c-41bc-9bbc-af4040b15ab8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:10.811666 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:10.811738 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f95f3c3-5f0c-41bc-9bbc-af4040b15ab8 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:10.818787 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 88/174] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:58:10 2022] GET /identity/v3/role_assignments?user.id=e6c6442222c54f34a84fdb4577045a9f&scope.project.id=b4cfdc9ec6a74df880eb9dcd5e97b98c&role.id=c426875a15834aa5b122e2a62267d15a => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:10.819517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95f3c3-5f0c-41bc-9bbc-af4040b15ab8 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:10.952000 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:10.952756 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d9caf-7cc0-4759-b799-73f00fe1ba89 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:10.974527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95f3c3-5f0c-41bc-9bbc-af4040b15ab8 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:11.019195 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a1dd89-d847-419a-ac53-4548ecaf1ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:11.020381 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a1dd89-d847-419a-ac53-4548ecaf1ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:11.020763 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a1dd89-d847-419a-ac53-4548ecaf1ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:11.021158 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a1dd89-d847-419a-ac53-4548ecaf1ca4 None admin] PATH_INFO: `/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:11.023494 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a1dd89-d847-419a-ac53-4548ecaf1ca4 None admin] RBAC: Authorizing `identity:get_project(project_id=b4cfdc9ec6a74df880eb9dcd5e97b98c)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:11.025746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a1dd89-d847-419a-ac53-4548ecaf1ca4 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:11.028555 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 87/175] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:10 2022] GET /identity/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:11.036127 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a1dd89-d847-419a-ac53-4548ecaf1ca4 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:11.039641 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee265328-d3fd-47e6-84af-94394ee17b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:11.040639 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee265328-d3fd-47e6-84af-94394ee17b51 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:11.044191 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee265328-d3fd-47e6-84af-94394ee17b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:11.044741 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee265328-d3fd-47e6-84af-94394ee17b51 None admin] PATH_INFO: `/v3/users/9c5fead1d17444a69ba60c5cf671f4c0` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:11.046430 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee265328-d3fd-47e6-84af-94394ee17b51 None admin] RBAC: Authorizing `identity:get_user(user_id=9c5fead1d17444a69ba60c5cf671f4c0)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:11.048230 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee265328-d3fd-47e6-84af-94394ee17b51 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:11.050926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 89/176] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:10 2022] GET /identity/v3/users/9c5fead1d17444a69ba60c5cf671f4c0 => generated 275 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:11.057458 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee265328-d3fd-47e6-84af-94394ee17b51 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:11.149150 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:11.182149 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a1dd89-d847-419a-ac53-4548ecaf1ca4 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:11.202528 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee265328-d3fd-47e6-84af-94394ee17b51 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:11.232861 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9714b860-660e-466f-a367-90adc073618b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:11.234068 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714b860-660e-466f-a367-90adc073618b None admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:11.234561 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714b860-660e-466f-a367-90adc073618b None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:11.235025 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714b860-660e-466f-a367-90adc073618b None admin] PATH_INFO: `/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c/users/f2a011a1ed924cb18c8a0c0f3bb6dcd4/roles/81d58cd096394c3aad885ad2d6f60f2c` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:11.239234 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9714b860-660e-466f-a367-90adc073618b None admin] RBAC: Authorizing `identity:create_grant(project_id=b4cfdc9ec6a74df880eb9dcd5e97b98c, user_id=f2a011a1ed924cb18c8a0c0f3bb6dcd4, role_id=81d58cd096394c3aad885ad2d6f60f2c)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:11.240953 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9714b860-660e-466f-a367-90adc073618b None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:11.261013 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 88/177] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 14:58:11 2022] PUT /identity/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c/users/f2a011a1ed924cb18c8a0c0f3bb6dcd4/roles/81d58cd096394c3aad885ad2d6f60f2c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 14:58:11.274493 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed96925-38c5-49a2-bf01-ee9ba3bcdfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:11.275269 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed96925-38c5-49a2-bf01-ee9ba3bcdfd4 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:11.275640 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed96925-38c5-49a2-bf01-ee9ba3bcdfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:11.275982 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed96925-38c5-49a2-bf01-ee9ba3bcdfd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:11.312901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:11.357176 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed96925-38c5-49a2-bf01-ee9ba3bcdfd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:11.358928 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed96925-38c5-49a2-bf01-ee9ba3bcdfd4 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:11.476122 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 90/178] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:58:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 14:58:11.482471 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9714b860-660e-466f-a367-90adc073618b None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:11.624540 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9714b860-660e-466f-a367-90adc073618b None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:11.671155 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96bab3-9049-4938-a987-0c01b0697ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:11.672155 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96bab3-9049-4938-a987-0c01b0697ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:11.672667 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96bab3-9049-4938-a987-0c01b0697ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:11.673117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96bab3-9049-4938-a987-0c01b0697ab1 None admin] PATH_INFO: `/v3/projects/2a392c80f2174a0083e4985ef028b967` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:11.674472 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96bab3-9049-4938-a987-0c01b0697ab1 None admin] RBAC: Authorizing `identity:get_project(project_id=2a392c80f2174a0083e4985ef028b967)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:11.676110 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96bab3-9049-4938-a987-0c01b0697ab1 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:11.677961 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 89/179] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:11 2022] GET /identity/v3/projects/2a392c80f2174a0083e4985ef028b967 => generated 312 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:11.684171 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed96925-38c5-49a2-bf01-ee9ba3bcdfd4 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:11.761697 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:11.806001 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed96925-38c5-49a2-bf01-ee9ba3bcdfd4 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:11.842641 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d532719-ac9b-4952-8f8f-17bca0c44298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:11.843263 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d532719-ac9b-4952-8f8f-17bca0c44298 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:11.843538 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d532719-ac9b-4952-8f8f-17bca0c44298 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:11.843811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d532719-ac9b-4952-8f8f-17bca0c44298 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:11.844550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d532719-ac9b-4952-8f8f-17bca0c44298 None admin] RBAC: Adding query filter params (role.id=3e3756d9482f4b9b86271eacdc19f523, scope.project.id=2a392c80f2174a0083e4985ef028b967, user.id=9c5fead1d17444a69ba60c5cf671f4c0) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:11.844855 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d532719-ac9b-4952-8f8f-17bca0c44298 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:11.846348 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d532719-ac9b-4952-8f8f-17bca0c44298 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:11.853490 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 91/180] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:58:11 2022] GET /identity/v3/role_assignments?user.id=9c5fead1d17444a69ba60c5cf671f4c0&scope.project.id=2a392c80f2174a0083e4985ef028b967&role.id=3e3756d9482f4b9b86271eacdc19f523 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:13.049597 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ad2ee-c5cb-48f5-8ef7-7162e524314e None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:13.050636 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ad2ee-c5cb-48f5-8ef7-7162e524314e None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:13.051102 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ad2ee-c5cb-48f5-8ef7-7162e524314e None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:13.052642 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 90/181] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:13.058418 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee3fdd-c96b-4dec-8775-959b7ebebff9 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:13.058883 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee3fdd-c96b-4dec-8775-959b7ebebff9 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:13.058981 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee3fdd-c96b-4dec-8775-959b7ebebff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:13.078181 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-abee3fdd-c96b-4dec-8775-959b7ebebff9 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:13.079389 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-abee3fdd-c96b-4dec-8775-959b7ebebff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:13.150518 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 92/182] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:13.410726 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a79b9f-e858-41a1-84c2-7cb1f484499a None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:13.410901 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a79b9f-e858-41a1-84c2-7cb1f484499a None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:13.411175 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a79b9f-e858-41a1-84c2-7cb1f484499a None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:13.412890 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 91/183] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:13.418734 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aafcdba-53ff-40a7-bbc5-b61756fc2209 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:13.419007 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aafcdba-53ff-40a7-bbc5-b61756fc2209 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:13.419280 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aafcdba-53ff-40a7-bbc5-b61756fc2209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:13.437424 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a79b9f-e858-41a1-84c2-7cb1f484499a None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:13.439460 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-8aafcdba-53ff-40a7-bbc5-b61756fc2209 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:13.440666 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:13.441648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-8aafcdba-53ff-40a7-bbc5-b61756fc2209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:13.502980 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:13.527414 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 93/184] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:13.556207 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eaf6f4-f9e7-409a-b40e-70608f91d400 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:13.556522 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eaf6f4-f9e7-409a-b40e-70608f91d400 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:13.556743 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eaf6f4-f9e7-409a-b40e-70608f91d400 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:13.558501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 94/185] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:58:13.563800 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d76d8-6ff5-419e-a4c6-0497bc4263ac None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:13.564079 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d76d8-6ff5-419e-a4c6-0497bc4263ac None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:13.564429 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d76d8-6ff5-419e-a4c6-0497bc4263ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:13.572308 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a79b9f-e858-41a1-84c2-7cb1f484499a None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:13.583222 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-da5d76d8-6ff5-419e-a4c6-0497bc4263ac None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:13.584501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-da5d76d8-6ff5-419e-a4c6-0497bc4263ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:13.611149 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85175820-9b7e-49e8-8f55-75ae444a8f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:13.611810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85175820-9b7e-49e8-8f55-75ae444a8f4c None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:13.612084 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85175820-9b7e-49e8-8f55-75ae444a8f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:13.612355 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85175820-9b7e-49e8-8f55-75ae444a8f4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:13.625863 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85175820-9b7e-49e8-8f55-75ae444a8f4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:13.631865 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-85175820-9b7e-49e8-8f55-75ae444a8f4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:58:13.631865 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:13.631865 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:13.631865 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:13.631865 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:13.631865 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:13.631865 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:13.631865 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:13.631865 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:13.631865 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:13.631865 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:13.631865 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:13.631865 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:13.631865 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:13.631865 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:13.631865 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:13.632815 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:13.632815 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:13.632815 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:13.632815 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:13.632815 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:13.632815 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:13.632815 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:13.632815 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:13.632815 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:13.632815 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:13.632815 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:13.632815 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:13.632815 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:13.632815 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:13.632815 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:13.632815 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:13.634868 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:13.634868 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:13.634868 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:13.634868 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:13.634868 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:13.634868 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:13.634868 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:13.634868 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:13.634868 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:13.634868 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:13.634868 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:58:13.634868 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:58:13.634868 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 92/186] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 14:58:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:58:13.644443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85175820-9b7e-49e8-8f55-75ae444a8f4c None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:13.670936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 95/187] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:58:13.715213 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:13.736759 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a9639-7aba-4fa8-a8d3-7aa831b2fc6c None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:13.737015 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a9639-7aba-4fa8-a8d3-7aa831b2fc6c None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:13.737369 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a9639-7aba-4fa8-a8d3-7aa831b2fc6c None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:13.738799 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 96/188] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:58:13.744661 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be756d-8729-4609-9775-6b2d8a234662 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:13.744935 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be756d-8729-4609-9775-6b2d8a234662 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:13.745233 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be756d-8729-4609-9775-6b2d8a234662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:13.759975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85175820-9b7e-49e8-8f55-75ae444a8f4c None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:13.766249 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-64be756d-8729-4609-9775-6b2d8a234662 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:13.767445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-64be756d-8729-4609-9775-6b2d8a234662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:13.805974 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c15d2e-b93d-489c-ba8e-01812d3da861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:13.806882 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c15d2e-b93d-489c-ba8e-01812d3da861 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:13.807571 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c15d2e-b93d-489c-ba8e-01812d3da861 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:13.808920 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c15d2e-b93d-489c-ba8e-01812d3da861 None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:13.809887 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c15d2e-b93d-489c-ba8e-01812d3da861 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:13.810364 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c15d2e-b93d-489c-ba8e-01812d3da861 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:13.811930 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c15d2e-b93d-489c-ba8e-01812d3da861 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:13.823464 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 93/189] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:13.829413 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c15d2e-b93d-489c-ba8e-01812d3da861 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:13.843487 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 97/190] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:13.844435 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be756d-8729-4609-9775-6b2d8a234662 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:13.890508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:13.939264 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c15d2e-b93d-489c-ba8e-01812d3da861 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:13.952639 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be756d-8729-4609-9775-6b2d8a234662 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:13.963782 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:13.975321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f5566-d94a-47c8-9f67-e3d4e594fef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:13.975853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f5566-d94a-47c8-9f67-e3d4e594fef2 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:13.976122 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f5566-d94a-47c8-9f67-e3d4e594fef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:13.976383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f5566-d94a-47c8-9f67-e3d4e594fef2 None admin] PATH_INFO: `/v3/users/25ef0f4f663742e3aca44ce3edb6c9f4` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:13.989050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d0acdd-70b5-4ab0-b064-c34181703c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:13.989822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0acdd-70b5-4ab0-b064-c34181703c85 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:13.990098 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0acdd-70b5-4ab0-b064-c34181703c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:13.990362 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0acdd-70b5-4ab0-b064-c34181703c85 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:13.992078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f5566-d94a-47c8-9f67-e3d4e594fef2 None admin] RBAC: Authorizing `identity:get_user(user_id=25ef0f4f663742e3aca44ce3edb6c9f4)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:13.993494 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f5566-d94a-47c8-9f67-e3d4e594fef2 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:13.995297 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 94/191] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:13 2022] GET /identity/v3/users/25ef0f4f663742e3aca44ce3edb6c9f4 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:13.995931 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f5566-d94a-47c8-9f67-e3d4e594fef2 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:14.005533 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d0acdd-70b5-4ab0-b064-c34181703c85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:14.011003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-b6d0acdd-70b5-4ab0-b064-c34181703c85 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:58:14.011003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:14.011003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:14.011003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:14.011003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:14.011003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:14.011003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:14.011003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:14.011003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:14.011003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:14.011003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:14.011003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:14.011003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:14.011003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:14.011003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:14.011003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:14.011995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:14.011995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:14.011995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:14.011995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:14.011995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:14.011995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:14.011995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:14.011995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:14.011995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:14.011995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:14.011995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:14.011995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:14.011995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:14.011995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:14.011995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:14.011995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:14.012856 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:14.012856 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:14.012856 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:14.012856 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:14.012856 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:14.012856 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:14.012856 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:14.012856 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:14.012856 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:14.012856 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:14.012856 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:58:14.012856 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:14.012856 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 98/192] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 14:58:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:14.013945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d0acdd-70b5-4ab0-b064-c34181703c85 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:14.102902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f5566-d94a-47c8-9f67-e3d4e594fef2 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:14.122366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d0acdd-70b5-4ab0-b064-c34181703c85 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:14.124386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:14.137340 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175baff-4f1a-49f2-bfd3-c79a211a26c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:14.138010 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175baff-4f1a-49f2-bfd3-c79a211a26c2 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:14.138395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175baff-4f1a-49f2-bfd3-c79a211a26c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:14.138654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175baff-4f1a-49f2-bfd3-c79a211a26c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:14.139753 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175baff-4f1a-49f2-bfd3-c79a211a26c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:14.141234 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175baff-4f1a-49f2-bfd3-c79a211a26c2 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:14.142798 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 95/193] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:58:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:14.143539 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175baff-4f1a-49f2-bfd3-c79a211a26c2 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:14.160125 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3117dce3-7ddd-4384-8621-3ab2b987d9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:14.160744 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117dce3-7ddd-4384-8621-3ab2b987d9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:14.161018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117dce3-7ddd-4384-8621-3ab2b987d9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:14.161321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117dce3-7ddd-4384-8621-3ab2b987d9d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:14.194705 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:14.231846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3117dce3-7ddd-4384-8621-3ab2b987d9d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:14.233352 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3117dce3-7ddd-4384-8621-3ab2b987d9d4 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:14.246164 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175baff-4f1a-49f2-bfd3-c79a211a26c2 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:14.282534 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24258b3b-d8c9-4a57-b6f5-d52135adb77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:14.283151 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24258b3b-d8c9-4a57-b6f5-d52135adb77e None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:14.283423 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24258b3b-d8c9-4a57-b6f5-d52135adb77e None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:14.283681 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24258b3b-d8c9-4a57-b6f5-d52135adb77e None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:14.284335 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24258b3b-d8c9-4a57-b6f5-d52135adb77e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:14.284631 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24258b3b-d8c9-4a57-b6f5-d52135adb77e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:14.286040 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24258b3b-d8c9-4a57-b6f5-d52135adb77e None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:14.294519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 96/194] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:58:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:14.295341 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24258b3b-d8c9-4a57-b6f5-d52135adb77e None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:14.338786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 99/195] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:58:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:58:14.339625 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3117dce3-7ddd-4384-8621-3ab2b987d9d4 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:14.341967 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:14.399483 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24258b3b-d8c9-4a57-b6f5-d52135adb77e None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:14.418360 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:14.442809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d172ec4-a616-4604-8606-7e3ab1740b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:14.443448 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d172ec4-a616-4604-8606-7e3ab1740b75 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:14.443720 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d172ec4-a616-4604-8606-7e3ab1740b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:14.443985 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d172ec4-a616-4604-8606-7e3ab1740b75 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:14.454699 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d172ec4-a616-4604-8606-7e3ab1740b75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:14.457339 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3117dce3-7ddd-4384-8621-3ab2b987d9d4 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:14.459893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-5d172ec4-a616-4604-8606-7e3ab1740b75 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:58:14.459893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:14.459893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:14.459893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:14.459893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:14.459893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:14.459893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:14.459893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:14.459893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:14.459893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:14.459893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:14.459893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:14.459893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:14.459893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:14.459893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:14.459893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:14.461279 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:14.461279 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:14.461279 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:14.461279 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:14.461279 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:14.461279 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:14.461279 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:14.461279 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:14.461279 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:14.461279 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:14.461279 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:14.461279 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:14.461279 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:14.461279 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:14.461279 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:14.461279 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:14.462829 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:14.462829 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:14.462829 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:14.462829 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:14.462829 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:14.462829 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:14.462829 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:14.462829 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:14.462829 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:14.462829 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:14.462829 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:58:14.462829 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:58:14.462829 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 97/196] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:14.462829 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d172ec4-a616-4604-8606-7e3ab1740b75 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:14.498598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c77631-ff5f-4fc3-a7c2-9e1ec1269b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:14.499229 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c77631-ff5f-4fc3-a7c2-9e1ec1269b82 None admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:14.499512 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c77631-ff5f-4fc3-a7c2-9e1ec1269b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:14.499773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c77631-ff5f-4fc3-a7c2-9e1ec1269b82 None admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:14.500418 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c77631-ff5f-4fc3-a7c2-9e1ec1269b82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:14.501901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c77631-ff5f-4fc3-a7c2-9e1ec1269b82 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:14.523657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 100/197] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 14:58:14 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:58:14.525345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c77631-ff5f-4fc3-a7c2-9e1ec1269b82 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:14.581568 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d172ec4-a616-4604-8606-7e3ab1740b75 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:14.604136 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:14.624976 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aea990-00e1-4f01-800c-c8ac15103b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:14.625669 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aea990-00e1-4f01-800c-c8ac15103b35 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:14.625949 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aea990-00e1-4f01-800c-c8ac15103b35 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:14.626225 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aea990-00e1-4f01-800c-c8ac15103b35 None admin] PATH_INFO: `/v3/users/25ef0f4f663742e3aca44ce3edb6c9f4` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:14.627502 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aea990-00e1-4f01-800c-c8ac15103b35 None admin] RBAC: Authorizing `identity:get_user(user_id=25ef0f4f663742e3aca44ce3edb6c9f4)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:14.629036 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aea990-00e1-4f01-800c-c8ac15103b35 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:14.636298 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 98/198] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:14 2022] GET /identity/v3/users/25ef0f4f663742e3aca44ce3edb6c9f4 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:14.636298 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aea990-00e1-4f01-800c-c8ac15103b35 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:14.667246 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c77631-ff5f-4fc3-a7c2-9e1ec1269b82 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:14.687314 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:14.712417 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b98851-a84a-424a-9e15-5055cf4584a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:14.712990 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b98851-a84a-424a-9e15-5055cf4584a0 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:14.713304 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b98851-a84a-424a-9e15-5055cf4584a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:14.713578 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b98851-a84a-424a-9e15-5055cf4584a0 None admin] PATH_INFO: `/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:14.714752 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b98851-a84a-424a-9e15-5055cf4584a0 None admin] RBAC: Authorizing `identity:get_project(project_id=5a38478a0f3f4ad4b28faa3b42446f00)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:14.716479 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b98851-a84a-424a-9e15-5055cf4584a0 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:14.723270 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 101/199] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:14 2022] GET /identity/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:14.723270 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b98851-a84a-424a-9e15-5055cf4584a0 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:14.757811 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aea990-00e1-4f01-800c-c8ac15103b35 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:14.800586 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d16fb5-b1eb-4102-a048-c7b8d30096bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:14.801494 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d16fb5-b1eb-4102-a048-c7b8d30096bb None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:14.802058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d16fb5-b1eb-4102-a048-c7b8d30096bb None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:14.802579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d16fb5-b1eb-4102-a048-c7b8d30096bb None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:14.803812 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d16fb5-b1eb-4102-a048-c7b8d30096bb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:14.803812 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d16fb5-b1eb-4102-a048-c7b8d30096bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:14.804941 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d16fb5-b1eb-4102-a048-c7b8d30096bb None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:14.815632 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 99/200] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 14:58:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:14.816481 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d16fb5-b1eb-4102-a048-c7b8d30096bb None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:14.853396 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b98851-a84a-424a-9e15-5055cf4584a0 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:14.887345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:14.897537 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98a337-20ab-4594-a7f0-3aecc56b65e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:14.898164 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98a337-20ab-4594-a7f0-3aecc56b65e2 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:14.898441 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98a337-20ab-4594-a7f0-3aecc56b65e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:14.898707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98a337-20ab-4594-a7f0-3aecc56b65e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:14.947769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d16fb5-b1eb-4102-a048-c7b8d30096bb None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:14.949919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:14.977837 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c98a337-20ab-4594-a7f0-3aecc56b65e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:14.979305 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c98a337-20ab-4594-a7f0-3aecc56b65e2 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:14.996420 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda439e4-f30b-4b8e-9c42-856f9092d4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:14.997379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda439e4-f30b-4b8e-9c42-856f9092d4df None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:14.997651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda439e4-f30b-4b8e-9c42-856f9092d4df None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:14.998306 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda439e4-f30b-4b8e-9c42-856f9092d4df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:14.999060 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda439e4-f30b-4b8e-9c42-856f9092d4df None admin] RBAC: Adding query filter params (role.id=c426875a15834aa5b122e2a62267d15a, scope.project.id=5a38478a0f3f4ad4b28faa3b42446f00, user.id=25ef0f4f663742e3aca44ce3edb6c9f4) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:14.999358 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda439e4-f30b-4b8e-9c42-856f9092d4df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:15.001354 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda439e4-f30b-4b8e-9c42-856f9092d4df None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:15.009861 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 100/201] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:58:14 2022] GET /identity/v3/role_assignments?user.id=25ef0f4f663742e3aca44ce3edb6c9f4&scope.project.id=5a38478a0f3f4ad4b28faa3b42446f00&role.id=c426875a15834aa5b122e2a62267d15a => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:15.010785 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda439e4-f30b-4b8e-9c42-856f9092d4df None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:15.114919 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 102/202] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:58:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:58:15.121466 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98a337-20ab-4594-a7f0-3aecc56b65e2 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:15.171692 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:15.188529 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda439e4-f30b-4b8e-9c42-856f9092d4df None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:15.272947 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98a337-20ab-4594-a7f0-3aecc56b65e2 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:15.288327 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e9478-87ac-41ea-8a81-b989ee08c3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:15.289913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e9478-87ac-41ea-8a81-b989ee08c3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:15.290404 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e9478-87ac-41ea-8a81-b989ee08c3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:15.290889 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e9478-87ac-41ea-8a81-b989ee08c3c8 None admin] PATH_INFO: `/v3/users/f2a011a1ed924cb18c8a0c0f3bb6dcd4` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:15.292945 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2e9478-87ac-41ea-8a81-b989ee08c3c8 None admin] RBAC: Authorizing `identity:get_user(user_id=f2a011a1ed924cb18c8a0c0f3bb6dcd4)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:15.295020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2e9478-87ac-41ea-8a81-b989ee08c3c8 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:15.297627 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 101/203] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:15 2022] GET /identity/v3/users/f2a011a1ed924cb18c8a0c0f3bb6dcd4 => generated 289 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:15.305688 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e9478-87ac-41ea-8a81-b989ee08c3c8 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:15.331283 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d74128a-95c9-4c14-88ab-0b11314dca25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:15.331935 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74128a-95c9-4c14-88ab-0b11314dca25 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:15.332233 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74128a-95c9-4c14-88ab-0b11314dca25 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:15.332480 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74128a-95c9-4c14-88ab-0b11314dca25 None admin] PATH_INFO: `/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:15.335118 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d74128a-95c9-4c14-88ab-0b11314dca25 None admin] RBAC: Authorizing `identity:get_project(project_id=5a38478a0f3f4ad4b28faa3b42446f00)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:15.336608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d74128a-95c9-4c14-88ab-0b11314dca25 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:15.340522 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 103/204] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:15 2022] GET /identity/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:15.346605 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d74128a-95c9-4c14-88ab-0b11314dca25 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:15.373321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:15.476150 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e9478-87ac-41ea-8a81-b989ee08c3c8 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:15.516308 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:15.528190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d74128a-95c9-4c14-88ab-0b11314dca25 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:15.546130 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c0153-cee6-42c2-946b-4a89dd0f7d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:15.546831 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c0153-cee6-42c2-946b-4a89dd0f7d98 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:15.547086 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c0153-cee6-42c2-946b-4a89dd0f7d98 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:15.547351 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c0153-cee6-42c2-946b-4a89dd0f7d98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:15.593776 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552e511-91c6-470b-969a-1f19278229c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:15.594396 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552e511-91c6-470b-969a-1f19278229c7 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:15.594674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552e511-91c6-470b-969a-1f19278229c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:15.594944 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552e511-91c6-470b-969a-1f19278229c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:15.595680 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d552e511-91c6-470b-969a-1f19278229c7 None admin] RBAC: Adding query filter params (role.id=b78105372b434d86872dd04e63dce750, scope.project.id=5a38478a0f3f4ad4b28faa3b42446f00, user.id=25ef0f4f663742e3aca44ce3edb6c9f4) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:15.595984 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d552e511-91c6-470b-969a-1f19278229c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:15.597870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d552e511-91c6-470b-969a-1f19278229c7 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:15.604078 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 104/205] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:58:15 2022] GET /identity/v3/role_assignments?user.id=25ef0f4f663742e3aca44ce3edb6c9f4&scope.project.id=5a38478a0f3f4ad4b28faa3b42446f00&role.id=b78105372b434d86872dd04e63dce750 => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:15.663436 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c0153-cee6-42c2-946b-4a89dd0f7d98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:15.665106 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c0153-cee6-42c2-946b-4a89dd0f7d98 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:15.706338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:15.788576 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 102/206] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:58:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 14:58:15.794545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552e511-91c6-470b-969a-1f19278229c7 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:15.950136 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552e511-91c6-470b-969a-1f19278229c7 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:16.001315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a85ce1-83cb-432d-853f-8f71253ac527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:16.002014 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a85ce1-83cb-432d-853f-8f71253ac527 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:16.002320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a85ce1-83cb-432d-853f-8f71253ac527 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:16.002593 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a85ce1-83cb-432d-853f-8f71253ac527 None admin] PATH_INFO: `/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:16.003907 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a85ce1-83cb-432d-853f-8f71253ac527 None admin] RBAC: Authorizing `identity:get_project(project_id=b4cfdc9ec6a74df880eb9dcd5e97b98c)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:16.005704 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a85ce1-83cb-432d-853f-8f71253ac527 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:16.007699 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 105/207] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:15 2022] GET /identity/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:16.014226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c0153-cee6-42c2-946b-4a89dd0f7d98 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:16.188353 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c0153-cee6-42c2-946b-4a89dd0f7d98 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:16.228351 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6cb5a-5532-48d7-bb13-ae8fc769da91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:16.229119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6cb5a-5532-48d7-bb13-ae8fc769da91 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:16.229492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6cb5a-5532-48d7-bb13-ae8fc769da91 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:16.229809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6cb5a-5532-48d7-bb13-ae8fc769da91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:16.230676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6cb5a-5532-48d7-bb13-ae8fc769da91 None admin] RBAC: Adding query filter params (role.id=81d58cd096394c3aad885ad2d6f60f2c, scope.project.id=b4cfdc9ec6a74df880eb9dcd5e97b98c, user.id=f2a011a1ed924cb18c8a0c0f3bb6dcd4) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:16.231010 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6cb5a-5532-48d7-bb13-ae8fc769da91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:16.232774 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6cb5a-5532-48d7-bb13-ae8fc769da91 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:16.234933 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:16.240781 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 103/208] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:58:16 2022] GET /identity/v3/role_assignments?user.id=f2a011a1ed924cb18c8a0c0f3bb6dcd4&scope.project.id=b4cfdc9ec6a74df880eb9dcd5e97b98c&role.id=81d58cd096394c3aad885ad2d6f60f2c => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:17.199270 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaed88f-9ad9-4679-b246-6dfdd14b1a1e None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:17.200319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaed88f-9ad9-4679-b246-6dfdd14b1a1e None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:17.200792 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaed88f-9ad9-4679-b246-6dfdd14b1a1e None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:17.202394 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 106/209] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:58:17.208278 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977bcd3e-1d36-48e5-9326-21131c8e8642 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:17.208848 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977bcd3e-1d36-48e5-9326-21131c8e8642 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:17.209369 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977bcd3e-1d36-48e5-9326-21131c8e8642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:17.228850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-977bcd3e-1d36-48e5-9326-21131c8e8642 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:17.230876 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-977bcd3e-1d36-48e5-9326-21131c8e8642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:17.328644 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 104/210] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:17.396671 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84110e63-b471-4794-9347-0e833b7b9f28 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:17.397324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84110e63-b471-4794-9347-0e833b7b9f28 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:17.397785 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84110e63-b471-4794-9347-0e833b7b9f28 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:17.399354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 107/211] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:17.405623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b8649-1475-46a2-87b7-c85d204e6565 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:17.405996 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b8649-1475-46a2-87b7-c85d204e6565 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:17.406277 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b8649-1475-46a2-87b7-c85d204e6565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:17.426273 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-719b8649-1475-46a2-87b7-c85d204e6565 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:17.427502 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-719b8649-1475-46a2-87b7-c85d204e6565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:17.508368 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 105/212] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:17.600389 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84110e63-b471-4794-9347-0e833b7b9f28 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:17.633429 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:17.649251 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb4a9d-f7c1-4ba2-91c4-2c742e67ac57 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:17.649587 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb4a9d-f7c1-4ba2-91c4-2c742e67ac57 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:17.649801 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb4a9d-f7c1-4ba2-91c4-2c742e67ac57 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:17.651526 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 106/213] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:17.657064 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd746874-9f00-4a9b-ab19-7e1d09f50741 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:17.657392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd746874-9f00-4a9b-ab19-7e1d09f50741 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:17.657695 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd746874-9f00-4a9b-ab19-7e1d09f50741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:17.685342 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-cd746874-9f00-4a9b-ab19-7e1d09f50741 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:17.686605 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-cd746874-9f00-4a9b-ab19-7e1d09f50741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:17.720849 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84110e63-b471-4794-9347-0e833b7b9f28 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:17.761266 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ecf7a-3c2d-4b54-a64b-fe952333ab46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:17.761938 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ecf7a-3c2d-4b54-a64b-fe952333ab46 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:17.762214 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ecf7a-3c2d-4b54-a64b-fe952333ab46 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:17.762560 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ecf7a-3c2d-4b54-a64b-fe952333ab46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:17.763836 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ecf7a-3c2d-4b54-a64b-fe952333ab46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:17.765432 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ecf7a-3c2d-4b54-a64b-fe952333ab46 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:17.766050 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:17.767306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 108/214] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:58:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:17.773557 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ecf7a-3c2d-4b54-a64b-fe952333ab46 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:17.789572 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 107/215] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:58:17.801162 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd746874-9f00-4a9b-ab19-7e1d09f50741 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:17.885938 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:17.894623 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ecf7a-3c2d-4b54-a64b-fe952333ab46 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:17.913986 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd746874-9f00-4a9b-ab19-7e1d09f50741 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:17.932593 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231bdd7-a8a6-4ddc-b070-f2caa46d9d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:17.933372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231bdd7-a8a6-4ddc-b070-f2caa46d9d92 None admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:17.933650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231bdd7-a8a6-4ddc-b070-f2caa46d9d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:17.933914 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231bdd7-a8a6-4ddc-b070-f2caa46d9d92 None admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:17.934756 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a231bdd7-a8a6-4ddc-b070-f2caa46d9d92 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:17.936226 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a231bdd7-a8a6-4ddc-b070-f2caa46d9d92 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:17.953805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4307614-40f3-40b0-a733-b1c8278be398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:17.954577 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4307614-40f3-40b0-a733-b1c8278be398 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:17.954863 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4307614-40f3-40b0-a733-b1c8278be398 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:17.955116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4307614-40f3-40b0-a733-b1c8278be398 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:17.966584 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 109/216] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 14:58:17 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 14:58:17.973773 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4307614-40f3-40b0-a733-b1c8278be398 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:17.979441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-f4307614-40f3-40b0-a733-b1c8278be398 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:17.979441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:17.979441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:17.979441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:17.979441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:17.979441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:17.979441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:17.979441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:17.979441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:17.979441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:17.979441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:17.979441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:17.979441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:17.979441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:17.979441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:17.979441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:17.980461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:17.980461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:17.980461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:17.980461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:17.980461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:17.980461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:17.980461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:17.980461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:17.980461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:17.980461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:17.980461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:17.980461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:17.980461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:17.980461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:17.980461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:17.980461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:17.981395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:17.981395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:17.981395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:17.981395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:17.981395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:17.981395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:17.981395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:17.981395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:17.981395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:17.981395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:17.981395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:17.981395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:58:17.984638 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 108/217] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:17.990801 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231bdd7-a8a6-4ddc-b070-f2caa46d9d92 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:18.070546 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4307614-40f3-40b0-a733-b1c8278be398 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:18.107741 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231bdd7-a8a6-4ddc-b070-f2caa46d9d92 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:18.128848 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:18.147964 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24421ec-bd26-4f35-ab16-c30b13c3bc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:18.148612 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24421ec-bd26-4f35-ab16-c30b13c3bc22 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:18.148886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24421ec-bd26-4f35-ab16-c30b13c3bc22 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:18.149181 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24421ec-bd26-4f35-ab16-c30b13c3bc22 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:18.150009 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24421ec-bd26-4f35-ab16-c30b13c3bc22 None admin] RBAC: Adding query filter params (name=member) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:18.150375 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24421ec-bd26-4f35-ab16-c30b13c3bc22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:18.152150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24421ec-bd26-4f35-ab16-c30b13c3bc22 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:18.162370 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 110/218] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 14:58:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:18.168854 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24421ec-bd26-4f35-ab16-c30b13c3bc22 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:18.186548 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:18.197771 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4307614-40f3-40b0-a733-b1c8278be398 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:18.246229 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51183c8-7ed2-4885-a738-a3c1941bf2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:18.246944 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51183c8-7ed2-4885-a738-a3c1941bf2c2 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:18.247243 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51183c8-7ed2-4885-a738-a3c1941bf2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:18.247952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51183c8-7ed2-4885-a738-a3c1941bf2c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:18.263793 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51183c8-7ed2-4885-a738-a3c1941bf2c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:18.269837 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-b51183c8-7ed2-4885-a738-a3c1941bf2c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:58:18.269837 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:18.269837 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:18.269837 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:18.269837 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:18.269837 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:18.269837 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:18.269837 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:18.269837 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:18.269837 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:18.269837 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:18.269837 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:18.269837 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:18.269837 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:18.269837 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:18.269837 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:18.270742 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:18.270742 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:18.270742 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:18.270742 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:18.270742 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:18.270742 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:18.270742 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:18.270742 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:18.270742 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:18.270742 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:18.270742 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:18.270742 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:18.270742 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:18.270742 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:18.270742 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:18.270742 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:18.271518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:18.271518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:18.271518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:18.271518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:18.271518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:18.271518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:18.271518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:18.271518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:18.271518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:18.271518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:18.271518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:58:18.271518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:58:18.271518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 109/219] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 14:58:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:18.276884 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51183c8-7ed2-4885-a738-a3c1941bf2c2 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:18.303022 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24421ec-bd26-4f35-ab16-c30b13c3bc22 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:18.347041 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d2831-81b5-4dfa-81dd-f96374e1adff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:18.348468 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d2831-81b5-4dfa-81dd-f96374e1adff None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:18.348937 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d2831-81b5-4dfa-81dd-f96374e1adff None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:18.349495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d2831-81b5-4dfa-81dd-f96374e1adff None admin] PATH_INFO: `/v3/users/627120f7c56849e3930279cda5727c09` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:18.365588 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d2831-81b5-4dfa-81dd-f96374e1adff None admin] RBAC: Authorizing `identity:get_user(user_id=627120f7c56849e3930279cda5727c09)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:18.367404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d2831-81b5-4dfa-81dd-f96374e1adff None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:18.369375 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 111/220] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:18 2022] GET /identity/v3/users/627120f7c56849e3930279cda5727c09 => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:18.370796 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d48517-5659-4744-9ff6-b431673338ad None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:18.371264 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d48517-5659-4744-9ff6-b431673338ad None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:18.371803 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d48517-5659-4744-9ff6-b431673338ad None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:18.377804 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 112/221] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:18.379079 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc33e1d-33be-40f7-9c50-86864eab0858 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:18.379361 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc33e1d-33be-40f7-9c50-86864eab0858 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:18.379629 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc33e1d-33be-40f7-9c50-86864eab0858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:18.399658 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-cdc33e1d-33be-40f7-9c50-86864eab0858 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:18.400859 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-cdc33e1d-33be-40f7-9c50-86864eab0858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:18.403320 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51183c8-7ed2-4885-a738-a3c1941bf2c2 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:18.414995 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:18.451808 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3882579-3f52-4a6c-9523-c29c9ee62296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:18.452435 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3882579-3f52-4a6c-9523-c29c9ee62296 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:18.453041 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3882579-3f52-4a6c-9523-c29c9ee62296 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:18.453041 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3882579-3f52-4a6c-9523-c29c9ee62296 None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:18.453681 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3882579-3f52-4a6c-9523-c29c9ee62296 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:18.453982 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3882579-3f52-4a6c-9523-c29c9ee62296 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:18.455390 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3882579-3f52-4a6c-9523-c29c9ee62296 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:18.469247 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 110/222] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:18.470499 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3882579-3f52-4a6c-9523-c29c9ee62296 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:18.485187 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:18.490527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 113/223] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:18.491656 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc33e1d-33be-40f7-9c50-86864eab0858 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:18.594939 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3882579-3f52-4a6c-9523-c29c9ee62296 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:18.601599 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc33e1d-33be-40f7-9c50-86864eab0858 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:18.640728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31d427-6760-444d-8499-59b4eaa2faad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:18.641922 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31d427-6760-444d-8499-59b4eaa2faad None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:18.642665 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31d427-6760-444d-8499-59b4eaa2faad None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:18.643791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31d427-6760-444d-8499-59b4eaa2faad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:18.665851 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f219b-cf86-40aa-b646-e77479936d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:18.666840 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f219b-cf86-40aa-b646-e77479936d17 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:18.667343 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f219b-cf86-40aa-b646-e77479936d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:18.667838 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f219b-cf86-40aa-b646-e77479936d17 None admin] PATH_INFO: `/v3/users/25ef0f4f663742e3aca44ce3edb6c9f4` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:18.669270 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f219b-cf86-40aa-b646-e77479936d17 None admin] RBAC: Authorizing `identity:get_user(user_id=25ef0f4f663742e3aca44ce3edb6c9f4)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:18.670923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f219b-cf86-40aa-b646-e77479936d17 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:18.672839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 114/224] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:18 2022] GET /identity/v3/users/25ef0f4f663742e3aca44ce3edb6c9f4 => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:18.679123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f219b-cf86-40aa-b646-e77479936d17 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:18.737076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:18.790867 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:18.797923 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a31d427-6760-444d-8499-59b4eaa2faad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:18.802968 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a31d427-6760-444d-8499-59b4eaa2faad None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:18.858409 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f219b-cf86-40aa-b646-e77479936d17 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:18.901986 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95534d7-0a28-4d55-afcd-ebf247405926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:18.902633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95534d7-0a28-4d55-afcd-ebf247405926 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:18.903488 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95534d7-0a28-4d55-afcd-ebf247405926 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:18.903766 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95534d7-0a28-4d55-afcd-ebf247405926 None admin] PATH_INFO: `/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:18.905204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95534d7-0a28-4d55-afcd-ebf247405926 None admin] RBAC: Authorizing `identity:get_project(project_id=5a38478a0f3f4ad4b28faa3b42446f00)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:18.906737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95534d7-0a28-4d55-afcd-ebf247405926 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:18.908459 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 115/225] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:18 2022] GET /identity/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:18.913887 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95534d7-0a28-4d55-afcd-ebf247405926 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:18.938622 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 111/226] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:58:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:58:18.939538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31d427-6760-444d-8499-59b4eaa2faad None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:19.011621 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:19.028341 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95534d7-0a28-4d55-afcd-ebf247405926 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:19.049414 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31d427-6760-444d-8499-59b4eaa2faad None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:19.070061 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af042a2-bec8-45a0-b9cc-65aa22f229e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:19.071002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af042a2-bec8-45a0-b9cc-65aa22f229e0 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:19.071461 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af042a2-bec8-45a0-b9cc-65aa22f229e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:19.071888 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af042a2-bec8-45a0-b9cc-65aa22f229e0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:19.075325 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:19.085654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af042a2-bec8-45a0-b9cc-65aa22f229e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:19.091781 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd20edc-87db-42fa-855b-63c31923416c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:19.092319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-6af042a2-bec8-45a0-b9cc-65aa22f229e0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:58:19.092319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:19.092319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:19.092319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:19.092319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:19.092319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:19.092319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:19.092319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:19.092319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:19.092319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:19.092319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:19.092319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:19.092319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:19.092319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:19.092319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:19.092319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:19.093382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:19.093382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:19.093382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:19.093382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:19.093382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:19.093382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:19.093382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:19.093382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:19.093382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:19.093382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:19.093382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:19.093382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:19.093382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:19.093382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:19.093382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:19.093382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:19.094333 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:19.094333 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:19.094333 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:19.094333 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:19.094333 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:19.094333 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:19.094333 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:19.094333 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:19.094333 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:19.094333 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:19.094333 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:58:19.094333 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:19.094934 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd20edc-87db-42fa-855b-63c31923416c None admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:19.094934 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd20edc-87db-42fa-855b-63c31923416c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:19.094934 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd20edc-87db-42fa-855b-63c31923416c None admin] PATH_INFO: `/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00/users/25ef0f4f663742e3aca44ce3edb6c9f4/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:19.094934 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd20edc-87db-42fa-855b-63c31923416c None admin] RBAC: Authorizing `identity:create_grant(project_id=5a38478a0f3f4ad4b28faa3b42446f00, user_id=25ef0f4f663742e3aca44ce3edb6c9f4, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:19.096379 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 116/227] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 14:58:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:19.096487 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd20edc-87db-42fa-855b-63c31923416c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:19.102873 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af042a2-bec8-45a0-b9cc-65aa22f229e0 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:19.112943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 112/228] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 14:58:18 2022] PUT /identity/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00/users/25ef0f4f663742e3aca44ce3edb6c9f4/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:58:19.114267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd20edc-87db-42fa-855b-63c31923416c None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:19.232358 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd20edc-87db-42fa-855b-63c31923416c None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:19.264248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:19.274327 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e5e19-abf2-4bc3-a81b-9ad9f0e971d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:19.275416 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e5e19-abf2-4bc3-a81b-9ad9f0e971d7 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:19.275914 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e5e19-abf2-4bc3-a81b-9ad9f0e971d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:19.276444 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e5e19-abf2-4bc3-a81b-9ad9f0e971d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:19.277381 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e5e19-abf2-4bc3-a81b-9ad9f0e971d7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:19.277933 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e5e19-abf2-4bc3-a81b-9ad9f0e971d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:19.280578 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e5e19-abf2-4bc3-a81b-9ad9f0e971d7 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:19.285942 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af042a2-bec8-45a0-b9cc-65aa22f229e0 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:19.293315 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 113/229] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:58:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:19.297055 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e5e19-abf2-4bc3-a81b-9ad9f0e971d7 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:19.374083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e3441-9b2d-4a8e-8b7f-bce7112659cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:19.374749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e3441-9b2d-4a8e-8b7f-bce7112659cb None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:19.374971 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e3441-9b2d-4a8e-8b7f-bce7112659cb None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:19.375234 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e3441-9b2d-4a8e-8b7f-bce7112659cb None admin] PATH_INFO: `/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:19.376526 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e3441-9b2d-4a8e-8b7f-bce7112659cb None admin] RBAC: Authorizing `identity:get_project(project_id=5a38478a0f3f4ad4b28faa3b42446f00)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:19.378016 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e3441-9b2d-4a8e-8b7f-bce7112659cb None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:19.379930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 117/230] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:19 2022] GET /identity/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00 => generated 302 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:19.386032 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e3441-9b2d-4a8e-8b7f-bce7112659cb None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:19.415985 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e5e19-abf2-4bc3-a81b-9ad9f0e971d7 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:19.456414 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57181f9-6446-4818-8252-a80271df255e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:19.457446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57181f9-6446-4818-8252-a80271df255e None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:19.457916 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57181f9-6446-4818-8252-a80271df255e None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:19.458373 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57181f9-6446-4818-8252-a80271df255e None admin] PATH_INFO: `/v3/users/25ef0f4f663742e3aca44ce3edb6c9f4` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:19.459759 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57181f9-6446-4818-8252-a80271df255e None admin] RBAC: Authorizing `identity:get_user(user_id=25ef0f4f663742e3aca44ce3edb6c9f4)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:19.461554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57181f9-6446-4818-8252-a80271df255e None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:19.463354 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 114/231] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:19 2022] GET /identity/v3/users/25ef0f4f663742e3aca44ce3edb6c9f4 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:19.470251 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57181f9-6446-4818-8252-a80271df255e None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:19.502667 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:19.556483 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:19.572196 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e3441-9b2d-4a8e-8b7f-bce7112659cb None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:19.626973 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57181f9-6446-4818-8252-a80271df255e None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:19.628652 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb92b1c-e622-4ee2-8322-ac565b0515c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:19.629376 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb92b1c-e622-4ee2-8322-ac565b0515c2 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:19.629650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb92b1c-e622-4ee2-8322-ac565b0515c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:19.629915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb92b1c-e622-4ee2-8322-ac565b0515c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:19.631313 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb92b1c-e622-4ee2-8322-ac565b0515c2 None admin] RBAC: Adding query filter params (role.id=3e3756d9482f4b9b86271eacdc19f523, scope.project.id=5a38478a0f3f4ad4b28faa3b42446f00, user.id=627120f7c56849e3930279cda5727c09) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:19.631313 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb92b1c-e622-4ee2-8322-ac565b0515c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:19.632509 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb92b1c-e622-4ee2-8322-ac565b0515c2 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:19.639277 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 118/232] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:58:19 2022] GET /identity/v3/role_assignments?user.id=627120f7c56849e3930279cda5727c09&scope.project.id=5a38478a0f3f4ad4b28faa3b42446f00&role.id=3e3756d9482f4b9b86271eacdc19f523 => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:19.665680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920e403-e9b4-47a9-94b3-5b5da293492c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:19.666694 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920e403-e9b4-47a9-94b3-5b5da293492c None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:19.667166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920e403-e9b4-47a9-94b3-5b5da293492c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:19.667690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920e403-e9b4-47a9-94b3-5b5da293492c None admin] PATH_INFO: `/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:19.669385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7920e403-e9b4-47a9-94b3-5b5da293492c None admin] RBAC: Authorizing `identity:get_project(project_id=5a38478a0f3f4ad4b28faa3b42446f00)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:19.671256 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7920e403-e9b4-47a9-94b3-5b5da293492c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:19.673741 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 115/233] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:19 2022] GET /identity/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:19.680706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb92b1c-e622-4ee2-8322-ac565b0515c2 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:19.823088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb92b1c-e622-4ee2-8322-ac565b0515c2 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:19.857259 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:19.873123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ad894-a1ce-453b-bcf6-cc737204a667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:19.873834 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ad894-a1ce-453b-bcf6-cc737204a667 None admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:19.874114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ad894-a1ce-453b-bcf6-cc737204a667 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:19.874382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ad894-a1ce-453b-bcf6-cc737204a667 None admin] PATH_INFO: `/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00/users/25ef0f4f663742e3aca44ce3edb6c9f4/roles/b78105372b434d86872dd04e63dce750` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:19.877120 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625ad894-a1ce-453b-bcf6-cc737204a667 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a38478a0f3f4ad4b28faa3b42446f00, user_id=25ef0f4f663742e3aca44ce3edb6c9f4, role_id=b78105372b434d86872dd04e63dce750)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:19.878586 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625ad894-a1ce-453b-bcf6-cc737204a667 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:19.899286 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 119/234] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 14:58:19 2022] PUT /identity/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00/users/25ef0f4f663742e3aca44ce3edb6c9f4/roles/b78105372b434d86872dd04e63dce750 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:58:20.739118 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2650d3-b0d5-4f5e-9335-3e39ecb25fb3 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:20.739607 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2650d3-b0d5-4f5e-9335-3e39ecb25fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:20.739607 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2650d3-b0d5-4f5e-9335-3e39ecb25fb3 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:20.741199 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 116/235] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:20.754898 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb33afb-66b5-4b48-894c-68446f64d858 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:20.755407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb33afb-66b5-4b48-894c-68446f64d858 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:20.755479 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb33afb-66b5-4b48-894c-68446f64d858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:20.784190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-4bb33afb-66b5-4b48-894c-68446f64d858 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:20.785455 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-4bb33afb-66b5-4b48-894c-68446f64d858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:20.789570 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd07c1-0659-436d-8e74-ca7580f59803 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:20.789850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd07c1-0659-436d-8e74-ca7580f59803 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:20.790122 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd07c1-0659-436d-8e74-ca7580f59803 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:20.791699 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 117/236] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:20.797808 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab9f99-2c97-4d3a-9073-9ff062d3511d None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:20.798090 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab9f99-2c97-4d3a-9073-9ff062d3511d None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:20.798359 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab9f99-2c97-4d3a-9073-9ff062d3511d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:20.829931 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-29ab9f99-2c97-4d3a-9073-9ff062d3511d None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:20.831127 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-29ab9f99-2c97-4d3a-9073-9ff062d3511d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:20.855775 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 120/237] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:58:20.956483 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 118/238] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:21.197601 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb33afb-66b5-4b48-894c-68446f64d858 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:21.329340 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab9f99-2c97-4d3a-9073-9ff062d3511d None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:21.332185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb33afb-66b5-4b48-894c-68446f64d858 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:21.352192 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:21.369613 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefeaddd-8b29-41b4-89af-80ff2fb143c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:21.370260 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefeaddd-8b29-41b4-89af-80ff2fb143c4 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:21.370538 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefeaddd-8b29-41b4-89af-80ff2fb143c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:21.370806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefeaddd-8b29-41b4-89af-80ff2fb143c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:21.371971 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefeaddd-8b29-41b4-89af-80ff2fb143c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:21.373450 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefeaddd-8b29-41b4-89af-80ff2fb143c4 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:21.375046 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 121/239] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:58:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:21.380636 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefeaddd-8b29-41b4-89af-80ff2fb143c4 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:21.485753 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefeaddd-8b29-41b4-89af-80ff2fb143c4 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:21.487694 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:21.525346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8c631-c8ba-4588-b9da-63fd244ebc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:21.529165 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8c631-c8ba-4588-b9da-63fd244ebc49 None admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:21.529165 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8c631-c8ba-4588-b9da-63fd244ebc49 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:21.529165 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8c631-c8ba-4588-b9da-63fd244ebc49 None admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:21.529165 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8c631-c8ba-4588-b9da-63fd244ebc49 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:21.529165 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8c631-c8ba-4588-b9da-63fd244ebc49 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:21.555517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 122/240] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 14:58:21 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:58:21.582142 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab9f99-2c97-4d3a-9073-9ff062d3511d None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:21.624829 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48d608-be22-4f99-9014-64a161f4f007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:21.625960 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48d608-be22-4f99-9014-64a161f4f007 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:21.626441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48d608-be22-4f99-9014-64a161f4f007 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:21.626903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48d608-be22-4f99-9014-64a161f4f007 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:21.639841 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca48d608-be22-4f99-9014-64a161f4f007 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:21.645288 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-ca48d608-be22-4f99-9014-64a161f4f007 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:58:21.645288 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:21.645288 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:21.645288 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:21.645288 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:21.645288 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:21.645288 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:21.645288 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:21.645288 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:21.645288 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:21.645288 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:21.645288 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:21.645288 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:21.645288 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:21.645288 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:21.645288 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:21.646259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:21.646259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:21.646259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:21.646259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:21.646259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:21.646259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:21.646259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:21.646259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:21.646259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:21.646259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:21.646259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:21.646259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:21.646259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:21.646259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:21.646259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:21.646259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:21.647128 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:21.647128 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:21.647128 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:21.647128 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:21.647128 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:21.647128 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:21.647128 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:21.647128 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:21.647128 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:21.647128 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:21.647128 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:58:21.647128 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:58:21.655156 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 119/241] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:21.658513 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8c631-c8ba-4588-b9da-63fd244ebc49 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:21.742467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:21.833519 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8c631-c8ba-4588-b9da-63fd244ebc49 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:21.874539 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53c1d2-7543-4ef7-b2dd-4e1c02d23d1a None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:21.875061 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53c1d2-7543-4ef7-b2dd-4e1c02d23d1a None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:21.875061 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53c1d2-7543-4ef7-b2dd-4e1c02d23d1a None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:21.876661 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 120/242] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:21.882198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54950cfb-8110-4d2c-aa29-a04f68503045 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:21.882464 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54950cfb-8110-4d2c-aa29-a04f68503045 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:21.882661 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b3d3f-5d9c-4a66-9aa8-946b834b9c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:21.883285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b3d3f-5d9c-4a66-9aa8-946b834b9c01 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:21.883379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54950cfb-8110-4d2c-aa29-a04f68503045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:21.883552 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b3d3f-5d9c-4a66-9aa8-946b834b9c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:21.883817 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b3d3f-5d9c-4a66-9aa8-946b834b9c01 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:21.884487 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068b3d3f-5d9c-4a66-9aa8-946b834b9c01 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:21.884782 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068b3d3f-5d9c-4a66-9aa8-946b834b9c01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:21.886191 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068b3d3f-5d9c-4a66-9aa8-946b834b9c01 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:21.895221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 123/243] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 14:58:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:21.900952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b3d3f-5d9c-4a66-9aa8-946b834b9c01 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:21.909492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-54950cfb-8110-4d2c-aa29-a04f68503045 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:21.910684 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-54950cfb-8110-4d2c-aa29-a04f68503045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:21.993562 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 121/244] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:22.011341 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b3d3f-5d9c-4a66-9aa8-946b834b9c01 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:22.013879 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:22.051719 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9747d8-8c00-48af-bdbd-9886e8826444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:22.052719 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9747d8-8c00-48af-bdbd-9886e8826444 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:22.053219 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9747d8-8c00-48af-bdbd-9886e8826444 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:22.053677 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9747d8-8c00-48af-bdbd-9886e8826444 None admin] PATH_INFO: `/v3/users/5c5383e4ffaf44319af42b3b11c5c0af` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:22.070950 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9747d8-8c00-48af-bdbd-9886e8826444 None admin] RBAC: Authorizing `identity:get_user(user_id=5c5383e4ffaf44319af42b3b11c5c0af)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:22.072751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9747d8-8c00-48af-bdbd-9886e8826444 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:22.074894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 124/245] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:21 2022] GET /identity/v3/users/5c5383e4ffaf44319af42b3b11c5c0af => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:22.080430 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54950cfb-8110-4d2c-aa29-a04f68503045 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:22.133209 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:22.186423 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54950cfb-8110-4d2c-aa29-a04f68503045 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:22.230486 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a60c7-33e7-4b52-8e02-db3eccdedffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:22.231219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a60c7-33e7-4b52-8e02-db3eccdedffa None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:22.231492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a60c7-33e7-4b52-8e02-db3eccdedffa None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:22.231760 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a60c7-33e7-4b52-8e02-db3eccdedffa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:22.324795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a60c7-33e7-4b52-8e02-db3eccdedffa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:22.326459 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a60c7-33e7-4b52-8e02-db3eccdedffa None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:22.380656 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9747d8-8c00-48af-bdbd-9886e8826444 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:22.385166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:22.437705 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 122/246] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:58:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:58:22.443704 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a60c7-33e7-4b52-8e02-db3eccdedffa None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:22.500390 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9747d8-8c00-48af-bdbd-9886e8826444 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:22.547823 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c88726-91a8-4698-87d2-6d483f1ffaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:22.548805 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c88726-91a8-4698-87d2-6d483f1ffaaf None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:22.549327 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c88726-91a8-4698-87d2-6d483f1ffaaf None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:22.550392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c88726-91a8-4698-87d2-6d483f1ffaaf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:22.564898 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a60c7-33e7-4b52-8e02-db3eccdedffa None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:22.565809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c88726-91a8-4698-87d2-6d483f1ffaaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:22.572572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-c4c88726-91a8-4698-87d2-6d483f1ffaaf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:58:22.572572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:22.572572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:22.572572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:22.572572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:22.572572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:22.572572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:22.572572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:22.572572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:22.572572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:22.572572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:22.572572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:22.572572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:22.572572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:22.572572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:22.572572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:22.573525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:22.573525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:22.573525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:22.573525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:22.573525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:22.573525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:22.573525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:22.573525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:22.573525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:22.573525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:22.573525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:22.573525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:22.573525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:22.573525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:22.573525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:22.573525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:22.574297 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:22.574297 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:22.574297 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:22.574297 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:22.574297 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:22.574297 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:22.574297 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:22.574297 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:22.574297 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:22.574297 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:22.574297 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:58:22.574297 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:22.576893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 125/247] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 14:58:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:58:22.578406 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49869e9-36e2-4585-9f12-3428e0b67f47 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:22.578857 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49869e9-36e2-4585-9f12-3428e0b67f47 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:22.579827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49869e9-36e2-4585-9f12-3428e0b67f47 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:22.581394 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 126/248] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:22.586069 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49869e9-36e2-4585-9f12-3428e0b67f47 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:22.630872 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:22.636022 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d85f0-c018-49b5-8ce7-488dbfaf0107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:22.636670 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d85f0-c018-49b5-8ce7-488dbfaf0107 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:22.636942 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d85f0-c018-49b5-8ce7-488dbfaf0107 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:22.637279 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d85f0-c018-49b5-8ce7-488dbfaf0107 None admin] PATH_INFO: `/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:22.639901 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d85f0-c018-49b5-8ce7-488dbfaf0107 None admin] RBAC: Authorizing `identity:get_project(project_id=5a38478a0f3f4ad4b28faa3b42446f00)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:22.642687 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d85f0-c018-49b5-8ce7-488dbfaf0107 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:22.648598 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 123/249] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:22 2022] GET /identity/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:22.650126 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ebf626-f3fe-48d7-b2d7-95f3461f02c7 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:22.650391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ebf626-f3fe-48d7-b2d7-95f3461f02c7 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:22.650664 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ebf626-f3fe-48d7-b2d7-95f3461f02c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:22.693489 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-00ebf626-f3fe-48d7-b2d7-95f3461f02c7 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:22.694685 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-00ebf626-f3fe-48d7-b2d7-95f3461f02c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:22.718863 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:22.787302 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49869e9-36e2-4585-9f12-3428e0b67f47 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:22.812351 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 124/250] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:22.813464 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ebf626-f3fe-48d7-b2d7-95f3461f02c7 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:22.837180 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1eb698-147a-478c-9847-ea5757f8ad6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:22.837814 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1eb698-147a-478c-9847-ea5757f8ad6c None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:22.838088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1eb698-147a-478c-9847-ea5757f8ad6c None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:22.838363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1eb698-147a-478c-9847-ea5757f8ad6c None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:22.839083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1eb698-147a-478c-9847-ea5757f8ad6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:22.839380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1eb698-147a-478c-9847-ea5757f8ad6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:22.840802 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1eb698-147a-478c-9847-ea5757f8ad6c None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:22.858780 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 127/251] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:22.864336 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8db62b-dab2-45eb-a563-cba641d22b93 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:22.864336 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8db62b-dab2-45eb-a563-cba641d22b93 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:22.864336 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8db62b-dab2-45eb-a563-cba641d22b93 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:22.864336 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 128/252] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:22.870162 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8db62b-dab2-45eb-a563-cba641d22b93 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:22.931475 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ebf626-f3fe-48d7-b2d7-95f3461f02c7 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:23.002016 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced7346-1927-4392-b84f-57a4a2a7a416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:23.002699 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced7346-1927-4392-b84f-57a4a2a7a416 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:23.002915 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced7346-1927-4392-b84f-57a4a2a7a416 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:23.003185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced7346-1927-4392-b84f-57a4a2a7a416 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:23.003925 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ced7346-1927-4392-b84f-57a4a2a7a416 None admin] RBAC: Adding query filter params (role.id=81d58cd096394c3aad885ad2d6f60f2c, scope.project.id=5a38478a0f3f4ad4b28faa3b42446f00, user.id=5c5383e4ffaf44319af42b3b11c5c0af) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:23.004233 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ced7346-1927-4392-b84f-57a4a2a7a416 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:23.005725 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ced7346-1927-4392-b84f-57a4a2a7a416 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:23.008797 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:23.011626 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 125/253] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:58:22 2022] GET /identity/v3/role_assignments?user.id=5c5383e4ffaf44319af42b3b11c5c0af&scope.project.id=5a38478a0f3f4ad4b28faa3b42446f00&role.id=81d58cd096394c3aad885ad2d6f60f2c => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:23.017917 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b1f21-c887-44aa-8b33-984406b5a1f5 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:23.018252 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b1f21-c887-44aa-8b33-984406b5a1f5 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:23.018474 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b1f21-c887-44aa-8b33-984406b5a1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:23.029095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8db62b-dab2-45eb-a563-cba641d22b93 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:23.056667 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-ec9b1f21-c887-44aa-8b33-984406b5a1f5 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:23.058521 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-ec9b1f21-c887-44aa-8b33-984406b5a1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:23.069647 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:23.071379 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f89478-5cc0-477a-8ff4-6f23d7f6c460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:23.072309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f89478-5cc0-477a-8ff4-6f23d7f6c460 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:23.072788 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f89478-5cc0-477a-8ff4-6f23d7f6c460 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:23.073315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f89478-5cc0-477a-8ff4-6f23d7f6c460 None admin] PATH_INFO: `/v3/users/25ef0f4f663742e3aca44ce3edb6c9f4` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:23.081162 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f89478-5cc0-477a-8ff4-6f23d7f6c460 None admin] RBAC: Authorizing `identity:get_user(user_id=25ef0f4f663742e3aca44ce3edb6c9f4)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:23.083163 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f89478-5cc0-477a-8ff4-6f23d7f6c460 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:23.085449 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 129/254] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:22 2022] GET /identity/v3/users/25ef0f4f663742e3aca44ce3edb6c9f4 => generated 283 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:23.091225 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f89478-5cc0-477a-8ff4-6f23d7f6c460 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:23.155617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 126/255] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:23.202833 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f89478-5cc0-477a-8ff4-6f23d7f6c460 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:23.263242 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819adc0-6f7b-417e-bd6f-5457466646dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:23.263890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819adc0-6f7b-417e-bd6f-5457466646dc None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:23.264166 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819adc0-6f7b-417e-bd6f-5457466646dc None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:23.264482 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819adc0-6f7b-417e-bd6f-5457466646dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:23.266589 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b1f21-c887-44aa-8b33-984406b5a1f5 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:23.307651 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:23.377354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819adc0-6f7b-417e-bd6f-5457466646dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:23.379068 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819adc0-6f7b-417e-bd6f-5457466646dc None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:23.473943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b1f21-c887-44aa-8b33-984406b5a1f5 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:23.490348 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 130/256] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:58:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:58:23.495499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819adc0-6f7b-417e-bd6f-5457466646dc None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:23.557529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:23.602407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c69f21-19d8-4af8-ac14-063f9d8321d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:23.603067 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c69f21-19d8-4af8-ac14-063f9d8321d9 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:23.603341 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c69f21-19d8-4af8-ac14-063f9d8321d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:23.603605 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c69f21-19d8-4af8-ac14-063f9d8321d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:23.607147 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819adc0-6f7b-417e-bd6f-5457466646dc None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:23.634561 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:23.643725 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c69f21-19d8-4af8-ac14-063f9d8321d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:23.644641 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd917b2-42e6-4d24-aae8-7dbe6a305d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:23.645319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd917b2-42e6-4d24-aae8-7dbe6a305d27 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:23.645600 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd917b2-42e6-4d24-aae8-7dbe6a305d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:23.645873 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd917b2-42e6-4d24-aae8-7dbe6a305d27 None admin] PATH_INFO: `/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:23.647068 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd917b2-42e6-4d24-aae8-7dbe6a305d27 None admin] RBAC: Authorizing `identity:get_project(project_id=5a38478a0f3f4ad4b28faa3b42446f00)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:23.648482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-13c69f21-19d8-4af8-ac14-063f9d8321d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:23.648482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:23.648482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:23.648482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:23.648482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:23.648482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:23.648482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:23.648482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:23.648482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:23.648482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:23.648482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:23.648482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:23.648482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:23.648482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:23.648482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:23.648482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:23.649356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:23.649356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:23.649356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:23.649356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:23.649356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:23.649356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:23.649356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:23.649356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:23.649356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:23.649356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:23.649356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:23.649356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:23.649356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:23.649356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:23.649356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:23.649356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:23.650730 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:23.650730 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:23.650730 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:23.650730 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:23.650730 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:23.650730 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:23.650730 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:23.650730 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:23.650730 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:23.650730 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:23.650730 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:23.650730 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:58:23.651316 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd917b2-42e6-4d24-aae8-7dbe6a305d27 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:23.651383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 127/257] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:23.651446 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 131/258] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:23 2022] GET /identity/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:23.655372 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c69f21-19d8-4af8-ac14-063f9d8321d9 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:23.659945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd917b2-42e6-4d24-aae8-7dbe6a305d27 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:23.774370 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd917b2-42e6-4d24-aae8-7dbe6a305d27 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:23.805583 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:23.822018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45be9d-e549-41fa-91a5-32e38d0914b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:23.822640 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45be9d-e549-41fa-91a5-32e38d0914b4 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:23.822917 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45be9d-e549-41fa-91a5-32e38d0914b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:23.823188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45be9d-e549-41fa-91a5-32e38d0914b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:23.824133 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad45be9d-e549-41fa-91a5-32e38d0914b4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:23.824437 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad45be9d-e549-41fa-91a5-32e38d0914b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:23.825953 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad45be9d-e549-41fa-91a5-32e38d0914b4 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:23.835561 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 132/259] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 14:58:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:23.836613 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45be9d-e549-41fa-91a5-32e38d0914b4 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:23.947961 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45be9d-e549-41fa-91a5-32e38d0914b4 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:23.959169 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c69f21-19d8-4af8-ac14-063f9d8321d9 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:24.006080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40853327-8e2f-48d6-a23c-ce88569da089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:24.007292 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40853327-8e2f-48d6-a23c-ce88569da089 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:24.007566 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40853327-8e2f-48d6-a23c-ce88569da089 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:24.007975 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40853327-8e2f-48d6-a23c-ce88569da089 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:24.008781 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40853327-8e2f-48d6-a23c-ce88569da089 None admin] RBAC: Adding query filter params (role.id=c426875a15834aa5b122e2a62267d15a, scope.project.id=5a38478a0f3f4ad4b28faa3b42446f00, user.id=25ef0f4f663742e3aca44ce3edb6c9f4) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:24.009090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40853327-8e2f-48d6-a23c-ce88569da089 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:24.010590 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40853327-8e2f-48d6-a23c-ce88569da089 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:24.018783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 133/260] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:58:23 2022] GET /identity/v3/role_assignments?user.id=25ef0f4f663742e3aca44ce3edb6c9f4&scope.project.id=5a38478a0f3f4ad4b28faa3b42446f00&role.id=c426875a15834aa5b122e2a62267d15a => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:24.019754 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40853327-8e2f-48d6-a23c-ce88569da089 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:24.073573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:24.079247 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c171195-c772-4159-a052-8ff756178c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:24.079964 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c171195-c772-4159-a052-8ff756178c5c None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:24.080245 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c171195-c772-4159-a052-8ff756178c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:24.080517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c171195-c772-4159-a052-8ff756178c5c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:24.112492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c171195-c772-4159-a052-8ff756178c5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:24.125723 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-2c171195-c772-4159-a052-8ff756178c5c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:58:24.125723 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:24.125723 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:24.125723 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:24.125723 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:24.125723 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:24.125723 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:24.125723 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:24.125723 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:24.125723 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:24.125723 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:24.125723 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:24.125723 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:24.125723 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:24.125723 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:24.125723 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:24.126680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:24.126680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:24.126680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:24.126680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:24.126680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:24.126680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:24.126680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:24.126680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:24.126680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:24.126680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:24.126680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:24.126680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:24.126680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:24.126680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:24.126680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:24.126680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:24.127447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:24.127447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:24.127447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:24.127447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:24.127447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:24.127447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:24.127447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:24.127447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:24.127447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:24.127447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:24.127447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:58:24.127447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:58:24.127447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 128/261] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 14:58:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 477 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:24.133697 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40853327-8e2f-48d6-a23c-ce88569da089 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:24.133959 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c171195-c772-4159-a052-8ff756178c5c None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:24.178087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a028d1-a5eb-4e4d-962e-83f3db7e6b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:24.178753 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a028d1-a5eb-4e4d-962e-83f3db7e6b9a None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:24.179032 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a028d1-a5eb-4e4d-962e-83f3db7e6b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:24.179295 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a028d1-a5eb-4e4d-962e-83f3db7e6b9a None admin] PATH_INFO: `/v3/users/627120f7c56849e3930279cda5727c09` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:24.180902 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a028d1-a5eb-4e4d-962e-83f3db7e6b9a None admin] RBAC: Authorizing `identity:get_user(user_id=627120f7c56849e3930279cda5727c09)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:24.182485 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a028d1-a5eb-4e4d-962e-83f3db7e6b9a None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:24.184278 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 134/262] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:24 2022] GET /identity/v3/users/627120f7c56849e3930279cda5727c09 => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:24.194429 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a028d1-a5eb-4e4d-962e-83f3db7e6b9a None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:24.260176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:24.333328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c171195-c772-4159-a052-8ff756178c5c None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:24.424685 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a028d1-a5eb-4e4d-962e-83f3db7e6b9a None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:24.429852 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07419016-b872-49d2-80f1-88363d68411c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:24.430741 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07419016-b872-49d2-80f1-88363d68411c None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:24.431216 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07419016-b872-49d2-80f1-88363d68411c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:24.431684 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07419016-b872-49d2-80f1-88363d68411c None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:24.432566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07419016-b872-49d2-80f1-88363d68411c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:24.433095 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07419016-b872-49d2-80f1-88363d68411c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:24.434630 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07419016-b872-49d2-80f1-88363d68411c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:24.441509 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:24.451352 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 129/263] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:58:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:24.460303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07419016-b872-49d2-80f1-88363d68411c None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:24.484236 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb7fe5-83ae-4454-bb55-ceed3a53e422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:24.485319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb7fe5-83ae-4454-bb55-ceed3a53e422 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:24.485809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb7fe5-83ae-4454-bb55-ceed3a53e422 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:24.486276 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb7fe5-83ae-4454-bb55-ceed3a53e422 None admin] PATH_INFO: `/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:24.488093 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb7fe5-83ae-4454-bb55-ceed3a53e422 None admin] RBAC: Authorizing `identity:get_project(project_id=5a38478a0f3f4ad4b28faa3b42446f00)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:24.489944 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb7fe5-83ae-4454-bb55-ceed3a53e422 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:24.497120 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 135/264] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:24 2022] GET /identity/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00 => generated 302 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:24.499454 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb7fe5-83ae-4454-bb55-ceed3a53e422 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:24.631676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07419016-b872-49d2-80f1-88363d68411c None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:24.671658 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb7fe5-83ae-4454-bb55-ceed3a53e422 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:24.677493 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:24.678698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adedf2cd-6ba8-4ead-a71a-87384e7bdd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:24.679347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adedf2cd-6ba8-4ead-a71a-87384e7bdd4b None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:24.679625 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adedf2cd-6ba8-4ead-a71a-87384e7bdd4b None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:24.680061 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adedf2cd-6ba8-4ead-a71a-87384e7bdd4b None admin] PATH_INFO: `/v3/users/25ef0f4f663742e3aca44ce3edb6c9f4` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:24.685396 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adedf2cd-6ba8-4ead-a71a-87384e7bdd4b None admin] RBAC: Authorizing `identity:get_user(user_id=25ef0f4f663742e3aca44ce3edb6c9f4)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:24.687620 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adedf2cd-6ba8-4ead-a71a-87384e7bdd4b None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:24.689329 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 130/265] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:24 2022] GET /identity/v3/users/25ef0f4f663742e3aca44ce3edb6c9f4 => generated 283 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:24.696532 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adedf2cd-6ba8-4ead-a71a-87384e7bdd4b None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:24.746481 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add84bcd-fddf-4cfb-a522-130f59a4fb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:24.747268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add84bcd-fddf-4cfb-a522-130f59a4fb6f None admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:24.747556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add84bcd-fddf-4cfb-a522-130f59a4fb6f None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:24.747833 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add84bcd-fddf-4cfb-a522-130f59a4fb6f None admin] PATH_INFO: `/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00/users/627120f7c56849e3930279cda5727c09/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:24.753238 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add84bcd-fddf-4cfb-a522-130f59a4fb6f None admin] RBAC: Authorizing `identity:create_grant(project_id=5a38478a0f3f4ad4b28faa3b42446f00, user_id=627120f7c56849e3930279cda5727c09, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:24.756071 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add84bcd-fddf-4cfb-a522-130f59a4fb6f None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:24.785723 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 136/266] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 14:58:24 2022] PUT /identity/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00/users/627120f7c56849e3930279cda5727c09/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:58:24.851913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adedf2cd-6ba8-4ead-a71a-87384e7bdd4b None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:24.890281 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b0b921-6d3d-4eb7-b0ad-6b0425169753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:24.890825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0b921-6d3d-4eb7-b0ad-6b0425169753 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:24.891107 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0b921-6d3d-4eb7-b0ad-6b0425169753 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:24.891378 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0b921-6d3d-4eb7-b0ad-6b0425169753 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:24.957053 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:24.969732 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b0b921-6d3d-4eb7-b0ad-6b0425169753 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:24.971300 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b0b921-6d3d-4eb7-b0ad-6b0425169753 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:25.088811 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 131/267] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:58:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 14:58:25.098727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add84bcd-fddf-4cfb-a522-130f59a4fb6f None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:25.157833 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:25.184137 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeceea94-80eb-4198-acb4-2038d66a3816 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:25.184448 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeceea94-80eb-4198-acb4-2038d66a3816 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:25.184687 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeceea94-80eb-4198-acb4-2038d66a3816 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:25.186138 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 132/268] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:25.192441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff0c4a-fe82-43f8-8ae8-37c3bb54f2c0 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:25.193052 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff0c4a-fe82-43f8-8ae8-37c3bb54f2c0 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:25.193567 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff0c4a-fe82-43f8-8ae8-37c3bb54f2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:25.214913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-f9ff0c4a-fe82-43f8-8ae8-37c3bb54f2c0 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:25.217220 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f9ff0c4a-fe82-43f8-8ae8-37c3bb54f2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:25.237709 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add84bcd-fddf-4cfb-a522-130f59a4fb6f None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:25.286874 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8142c96b-873a-4c03-bb2c-6d5e4300cf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:25.287754 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142c96b-873a-4c03-bb2c-6d5e4300cf23 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:25.288100 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142c96b-873a-4c03-bb2c-6d5e4300cf23 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:25.288439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142c96b-873a-4c03-bb2c-6d5e4300cf23 None admin] PATH_INFO: `/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:25.289827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8142c96b-873a-4c03-bb2c-6d5e4300cf23 None admin] RBAC: Authorizing `identity:get_project(project_id=5a38478a0f3f4ad4b28faa3b42446f00)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:25.291369 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8142c96b-873a-4c03-bb2c-6d5e4300cf23 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:25.297926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 137/269] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:25 2022] GET /identity/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:25.297926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c0288-be07-4b22-89bd-b85c49f6d5eb None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:25.297926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c0288-be07-4b22-89bd-b85c49f6d5eb None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:25.297926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c0288-be07-4b22-89bd-b85c49f6d5eb None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:25.297926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 138/270] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:25.304541 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c0288-be07-4b22-89bd-b85c49f6d5eb None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:25.319146 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 133/271] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:58:25.326921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0bfbf-84d1-4aed-9468-2b5ae4e351f2 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:25.327430 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0bfbf-84d1-4aed-9468-2b5ae4e351f2 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:25.327866 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0bfbf-84d1-4aed-9468-2b5ae4e351f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:25.357502 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-08a0bfbf-84d1-4aed-9468-2b5ae4e351f2 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:25.359027 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-08a0bfbf-84d1-4aed-9468-2b5ae4e351f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:25.390911 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:25.433185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c0288-be07-4b22-89bd-b85c49f6d5eb None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:25.446226 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:25.474675 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57893073-7922-4b59-8d5f-de03264507a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:25.475642 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57893073-7922-4b59-8d5f-de03264507a5 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:25.476119 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57893073-7922-4b59-8d5f-de03264507a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:25.476603 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57893073-7922-4b59-8d5f-de03264507a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:25.477572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57893073-7922-4b59-8d5f-de03264507a5 None admin] RBAC: Adding query filter params (role.id=b78105372b434d86872dd04e63dce750, scope.project.id=5a38478a0f3f4ad4b28faa3b42446f00, user.id=25ef0f4f663742e3aca44ce3edb6c9f4) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:25.478076 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57893073-7922-4b59-8d5f-de03264507a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:25.479864 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57893073-7922-4b59-8d5f-de03264507a5 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:25.487411 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 139/272] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:58:25 2022] GET /identity/v3/role_assignments?user.id=25ef0f4f663742e3aca44ce3edb6c9f4&scope.project.id=5a38478a0f3f4ad4b28faa3b42446f00&role.id=b78105372b434d86872dd04e63dce750 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:25.525215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 134/273] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:25.740262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57893073-7922-4b59-8d5f-de03264507a5 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:25.854535 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a0bfbf-84d1-4aed-9468-2b5ae4e351f2 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:25.861672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57893073-7922-4b59-8d5f-de03264507a5 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:25.926463 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8642c-cee4-4135-93f7-151e0c3c6f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:25.927122 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8642c-cee4-4135-93f7-151e0c3c6f37 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:25.927395 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8642c-cee4-4135-93f7-151e0c3c6f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:25.927662 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8642c-cee4-4135-93f7-151e0c3c6f37 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:25.949012 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad8642c-cee4-4135-93f7-151e0c3c6f37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:25.954465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-fad8642c-cee4-4135-93f7-151e0c3c6f37 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:25.954465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:25.954465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:25.954465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:25.954465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:25.954465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:25.954465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:25.954465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:25.954465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:25.954465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:25.954465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:25.954465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:25.954465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:25.954465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:25.954465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:25.954465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:25.955427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:25.955427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:25.955427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:25.955427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:25.955427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:25.955427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:25.955427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:25.955427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:25.955427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:25.955427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:25.955427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:25.955427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:25.955427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:25.955427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:25.955427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:25.955427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:25.956222 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:25.956222 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:25.956222 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:25.956222 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:25.956222 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:25.956222 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:25.956222 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:25.956222 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:25.956222 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:25.956222 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:25.956222 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:25.956222 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:25.956222 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 140/274] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:25.961881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8642c-cee4-4135-93f7-151e0c3c6f37 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:25.999562 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a0bfbf-84d1-4aed-9468-2b5ae4e351f2 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:26.018105 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:26.062915 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca26c64-44e5-4c31-ba6c-d575a2144a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:26.063841 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca26c64-44e5-4c31-ba6c-d575a2144a8b None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:26.065913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca26c64-44e5-4c31-ba6c-d575a2144a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:26.066326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca26c64-44e5-4c31-ba6c-d575a2144a8b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:26.067638 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca26c64-44e5-4c31-ba6c-d575a2144a8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:26.069190 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca26c64-44e5-4c31-ba6c-d575a2144a8b None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:26.071081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 135/275] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:58:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:26.072243 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0ddae-7d54-4d07-baa6-d00396994b10 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:26.072521 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0ddae-7d54-4d07-baa6-d00396994b10 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:26.072783 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0ddae-7d54-4d07-baa6-d00396994b10 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:26.077396 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 136/276] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:26.077396 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea0ddae-7d54-4d07-baa6-d00396994b10 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:26.104094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:26.116867 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8642c-cee4-4135-93f7-151e0c3c6f37 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:26.160184 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c9f505-0bee-4c1c-8631-d542339fc50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:26.160900 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9f505-0bee-4c1c-8631-d542339fc50b None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:26.161203 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9f505-0bee-4c1c-8631-d542339fc50b None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:26.161634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9f505-0bee-4c1c-8631-d542339fc50b None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:26.162307 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c9f505-0bee-4c1c-8631-d542339fc50b None admin] RBAC: Adding query filter params (name=member) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:26.165845 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c9f505-0bee-4c1c-8631-d542339fc50b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:26.165845 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c9f505-0bee-4c1c-8631-d542339fc50b None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:26.180747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 141/277] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 14:58:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:26.180747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295718e-84b6-4ac3-bba9-ab87b39ec47c None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:26.180747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295718e-84b6-4ac3-bba9-ab87b39ec47c None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:26.180747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295718e-84b6-4ac3-bba9-ab87b39ec47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:26.205246 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea0ddae-7d54-4d07-baa6-d00396994b10 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:26.205703 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-7295718e-84b6-4ac3-bba9-ab87b39ec47c None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:26.207265 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-7295718e-84b6-4ac3-bba9-ab87b39ec47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:26.246923 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2a957-4528-4482-b693-8f0fbf949e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:26.247804 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2a957-4528-4482-b693-8f0fbf949e4a None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:26.248082 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2a957-4528-4482-b693-8f0fbf949e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:26.248354 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2a957-4528-4482-b693-8f0fbf949e4a None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:26.249017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d2a957-4528-4482-b693-8f0fbf949e4a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:26.253460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d2a957-4528-4482-b693-8f0fbf949e4a None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:26.284168 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 137/278] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 14:58:26 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 14:58:26.285961 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2a957-4528-4482-b693-8f0fbf949e4a None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:26.310776 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 142/279] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:26.408583 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:26.439301 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2a957-4528-4482-b693-8f0fbf949e4a None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:26.483440 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df22ae-37fd-4a82-a299-44350df9cd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:26.484087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df22ae-37fd-4a82-a299-44350df9cd64 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:26.484364 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df22ae-37fd-4a82-a299-44350df9cd64 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:26.484641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df22ae-37fd-4a82-a299-44350df9cd64 None admin] PATH_INFO: `/v3/users/48830eecc6a54aa6bad6b8d4e33c1684` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:26.499853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df22ae-37fd-4a82-a299-44350df9cd64 None admin] RBAC: Authorizing `identity:get_user(user_id=48830eecc6a54aa6bad6b8d4e33c1684)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:26.501495 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df22ae-37fd-4a82-a299-44350df9cd64 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:26.505256 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 138/280] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:26 2022] GET /identity/v3/users/48830eecc6a54aa6bad6b8d4e33c1684 => generated 293 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:26.509474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295718e-84b6-4ac3-bba9-ab87b39ec47c None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:26.544148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:26.589075 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df22ae-37fd-4a82-a299-44350df9cd64 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:26.631745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295718e-84b6-4ac3-bba9-ab87b39ec47c None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:26.677000 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f889832-a134-4568-9c7f-a0db05c74da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:26.677660 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f889832-a134-4568-9c7f-a0db05c74da2 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:26.677936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f889832-a134-4568-9c7f-a0db05c74da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:26.678220 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f889832-a134-4568-9c7f-a0db05c74da2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:26.679000 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f889832-a134-4568-9c7f-a0db05c74da2 None admin] RBAC: Adding query filter params (role.id=3e3756d9482f4b9b86271eacdc19f523, user.id=48830eecc6a54aa6bad6b8d4e33c1684, scope.system=all) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:26.679305 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f889832-a134-4568-9c7f-a0db05c74da2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:26.680944 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f889832-a134-4568-9c7f-a0db05c74da2 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:26.691536 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 143/281] 10.222.0.44 () {56 vars in 1308 bytes} [Sun Sep 11 14:58:26 2022] GET /identity/v3/role_assignments?user.id=48830eecc6a54aa6bad6b8d4e33c1684&scope.system=all&role.id=3e3756d9482f4b9b86271eacdc19f523 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:26.720801 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df22ae-37fd-4a82-a299-44350df9cd64 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:26.764853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:26.772845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf37fea-b4ad-4c73-b4b8-030246b52d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:26.774522 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf37fea-b4ad-4c73-b4b8-030246b52d8d None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:26.775450 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf37fea-b4ad-4c73-b4b8-030246b52d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:26.776161 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf37fea-b4ad-4c73-b4b8-030246b52d8d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:26.791983 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf37fea-b4ad-4c73-b4b8-030246b52d8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:26.798382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-dcf37fea-b4ad-4c73-b4b8-030246b52d8d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:58:26.798382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:26.798382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:26.798382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:26.798382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:26.798382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:26.798382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:26.798382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:26.798382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:26.798382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:26.798382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:26.798382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:26.798382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:26.798382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:26.798382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:26.798382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:26.799504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:26.799504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:26.799504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:26.799504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:26.799504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:26.799504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:26.799504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:26.799504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:26.799504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:26.799504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:26.799504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:26.799504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:26.799504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:26.799504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:26.799504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:26.799504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:26.800358 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:26.800358 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:26.800358 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:26.800358 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:26.800358 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:26.800358 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:26.800358 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:26.800358 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:26.800358 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:26.800358 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:26.800358 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:58:26.800358 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:58:26.804086 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 139/282] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:58:26.814325 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f889832-a134-4568-9c7f-a0db05c74da2 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:26.948976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:26.970971 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f889832-a134-4568-9c7f-a0db05c74da2 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:27.011810 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362106b1-644e-405d-8657-33e4b490a476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:27.012478 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362106b1-644e-405d-8657-33e4b490a476 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:27.012745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362106b1-644e-405d-8657-33e4b490a476 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:27.013009 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362106b1-644e-405d-8657-33e4b490a476 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:27.013723 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362106b1-644e-405d-8657-33e4b490a476 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:27.014016 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362106b1-644e-405d-8657-33e4b490a476 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:27.015435 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362106b1-644e-405d-8657-33e4b490a476 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:27.024710 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 144/283] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 14:58:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:27.030466 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf37fea-b4ad-4c73-b4b8-030246b52d8d None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:27.210488 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf37fea-b4ad-4c73-b4b8-030246b52d8d None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:27.267190 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84fc50e-60e7-4f36-82b4-b2f45e965f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:27.267776 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fc50e-60e7-4f36-82b4-b2f45e965f6d None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:27.268056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fc50e-60e7-4f36-82b4-b2f45e965f6d None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:27.268323 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fc50e-60e7-4f36-82b4-b2f45e965f6d None admin] PATH_INFO: `/v3/users/5c5383e4ffaf44319af42b3b11c5c0af` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:27.269610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84fc50e-60e7-4f36-82b4-b2f45e965f6d None admin] RBAC: Authorizing `identity:get_user(user_id=5c5383e4ffaf44319af42b3b11c5c0af)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:27.273855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84fc50e-60e7-4f36-82b4-b2f45e965f6d None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:27.280417 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 140/284] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:27 2022] GET /identity/v3/users/5c5383e4ffaf44319af42b3b11c5c0af => generated 297 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:27.282316 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362106b1-644e-405d-8657-33e4b490a476 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:27.391845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28207cb5-e540-4f08-8f64-d0809904ae63 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:27.392174 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28207cb5-e540-4f08-8f64-d0809904ae63 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:27.392504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28207cb5-e540-4f08-8f64-d0809904ae63 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:27.395538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 141/285] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:27.404531 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43546b29-795d-4642-84d2-bad88f6f881a None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:27.404903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43546b29-795d-4642-84d2-bad88f6f881a None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:27.405385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43546b29-795d-4642-84d2-bad88f6f881a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:27.434205 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-43546b29-795d-4642-84d2-bad88f6f881a None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:27.435627 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-43546b29-795d-4642-84d2-bad88f6f881a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:27.456130 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362106b1-644e-405d-8657-33e4b490a476 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:27.508583 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e75c12-4cf6-47cc-a800-fde086b4be4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:27.509311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e75c12-4cf6-47cc-a800-fde086b4be4f None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:27.509594 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e75c12-4cf6-47cc-a800-fde086b4be4f None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:27.513381 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e75c12-4cf6-47cc-a800-fde086b4be4f None admin] PATH_INFO: `/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:27.520090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e75c12-4cf6-47cc-a800-fde086b4be4f None admin] RBAC: Authorizing `identity:get_project(project_id=5a38478a0f3f4ad4b28faa3b42446f00)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:27.523254 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e75c12-4cf6-47cc-a800-fde086b4be4f None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:27.531670 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 145/286] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:27 2022] GET /identity/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00 => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:27.537354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e75c12-4cf6-47cc-a800-fde086b4be4f None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:27.561228 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:27.578256 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:27.585930 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 142/287] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:58:27.673686 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e75c12-4cf6-47cc-a800-fde086b4be4f None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:27.717705 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058da66c-d3ce-4351-8e10-405f3b19e24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:27.718836 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058da66c-d3ce-4351-8e10-405f3b19e24a None admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:27.719321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058da66c-d3ce-4351-8e10-405f3b19e24a None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:27.719803 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058da66c-d3ce-4351-8e10-405f3b19e24a None admin] PATH_INFO: `/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00/users/5c5383e4ffaf44319af42b3b11c5c0af/roles/81d58cd096394c3aad885ad2d6f60f2c` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:27.721987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058da66c-d3ce-4351-8e10-405f3b19e24a None admin] RBAC: Authorizing `identity:create_grant(project_id=5a38478a0f3f4ad4b28faa3b42446f00, user_id=5c5383e4ffaf44319af42b3b11c5c0af, role_id=81d58cd096394c3aad885ad2d6f60f2c)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:27.724109 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058da66c-d3ce-4351-8e10-405f3b19e24a None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:27.743150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 146/288] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 14:58:27 2022] PUT /identity/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00/users/5c5383e4ffaf44319af42b3b11c5c0af/roles/81d58cd096394c3aad885ad2d6f60f2c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 14:58:27.942597 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43546b29-795d-4642-84d2-bad88f6f881a None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:28.090697 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43546b29-795d-4642-84d2-bad88f6f881a None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:28.128545 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea299153-d1b3-4176-9614-e0e95e6436e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:28.129230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea299153-d1b3-4176-9614-e0e95e6436e6 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:28.129506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea299153-d1b3-4176-9614-e0e95e6436e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:28.129772 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea299153-d1b3-4176-9614-e0e95e6436e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:28.146003 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea299153-d1b3-4176-9614-e0e95e6436e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:28.151538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-ea299153-d1b3-4176-9614-e0e95e6436e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:28.151538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:28.151538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:28.151538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:28.151538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:28.151538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:28.151538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:28.151538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:28.151538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:28.151538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:28.151538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:28.151538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:28.151538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:28.151538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:28.151538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:28.151538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:28.153000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:28.153000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:28.153000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:28.153000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:28.153000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:28.153000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:28.153000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:28.153000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:28.153000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:28.153000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:28.153000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:28.153000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:28.153000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:28.153000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:28.153000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:28.153000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:28.154362 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:28.154362 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:28.154362 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:28.154362 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:28.154362 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:28.154362 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:28.154362 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:28.154362 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:28.154362 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:28.154362 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:28.154362 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:28.154362 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:58:28.154362 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 143/289] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:28.163378 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058da66c-d3ce-4351-8e10-405f3b19e24a None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:28.295180 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:28.315983 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058da66c-d3ce-4351-8e10-405f3b19e24a None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:28.351563 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737fc08-ee35-4e4c-9eaa-46945e65a307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:28.352243 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737fc08-ee35-4e4c-9eaa-46945e65a307 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:28.352556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737fc08-ee35-4e4c-9eaa-46945e65a307 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:28.352825 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737fc08-ee35-4e4c-9eaa-46945e65a307 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:28.353525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737fc08-ee35-4e4c-9eaa-46945e65a307 None admin] RBAC: Adding query filter params (name=member) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:28.353829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737fc08-ee35-4e4c-9eaa-46945e65a307 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:28.355224 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737fc08-ee35-4e4c-9eaa-46945e65a307 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:28.365722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 147/290] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 14:58:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:28.374293 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea299153-d1b3-4176-9614-e0e95e6436e6 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:28.449541 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:28.514751 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea299153-d1b3-4176-9614-e0e95e6436e6 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:28.563917 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8d335-b3c5-4290-9032-7cca82e2627d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:28.564492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8d335-b3c5-4290-9032-7cca82e2627d None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:28.564767 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8d335-b3c5-4290-9032-7cca82e2627d None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:28.565037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8d335-b3c5-4290-9032-7cca82e2627d None admin] PATH_INFO: `/v3/users/627120f7c56849e3930279cda5727c09` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:28.566225 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8d335-b3c5-4290-9032-7cca82e2627d None admin] RBAC: Authorizing `identity:get_user(user_id=627120f7c56849e3930279cda5727c09)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:28.567692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8d335-b3c5-4290-9032-7cca82e2627d None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:28.569472 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 144/291] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:28 2022] GET /identity/v3/users/627120f7c56849e3930279cda5727c09 => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:28.575450 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737fc08-ee35-4e4c-9eaa-46945e65a307 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:28.709551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737fc08-ee35-4e4c-9eaa-46945e65a307 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:28.753466 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e735b-7630-4443-b330-d70255a89a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:28.754092 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e735b-7630-4443-b330-d70255a89a8d None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:28.754457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e735b-7630-4443-b330-d70255a89a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:28.754730 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e735b-7630-4443-b330-d70255a89a8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:28.774892 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:28.835104 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375e735b-7630-4443-b330-d70255a89a8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:28.837709 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375e735b-7630-4443-b330-d70255a89a8d None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:28.965646 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 148/292] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:58:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:58:28.971539 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8d335-b3c5-4290-9032-7cca82e2627d None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:29.115159 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8d335-b3c5-4290-9032-7cca82e2627d None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:29.140462 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:29.167027 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89aa03-c7ad-42a9-ad58-8aba03c04220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:29.167721 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89aa03-c7ad-42a9-ad58-8aba03c04220 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:29.168002 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89aa03-c7ad-42a9-ad58-8aba03c04220 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:29.168275 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89aa03-c7ad-42a9-ad58-8aba03c04220 None admin] PATH_INFO: `/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:29.169860 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e89aa03-c7ad-42a9-ad58-8aba03c04220 None admin] RBAC: Authorizing `identity:get_project(project_id=5a38478a0f3f4ad4b28faa3b42446f00)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:29.171799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e89aa03-c7ad-42a9-ad58-8aba03c04220 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:29.173862 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 145/293] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:28 2022] GET /identity/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:29.180492 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e735b-7630-4443-b330-d70255a89a8d None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:29.213443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2cba5-bed4-48a5-8dc1-203b5f390eb8 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:29.213789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2cba5-bed4-48a5-8dc1-203b5f390eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:29.213980 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2cba5-bed4-48a5-8dc1-203b5f390eb8 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:29.215482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 146/294] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:29.226337 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5d1c3-2678-4840-9f65-34ce542cf40c None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:29.226672 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5d1c3-2678-4840-9f65-34ce542cf40c None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:29.226868 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5d1c3-2678-4840-9f65-34ce542cf40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:29.265339 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-a5f5d1c3-2678-4840-9f65-34ce542cf40c None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:29.266591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-a5f5d1c3-2678-4840-9f65-34ce542cf40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:29.277701 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:29.375380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e735b-7630-4443-b330-d70255a89a8d None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:29.385637 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 147/295] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:29.391201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18218ed1-8b7b-42bb-a3e6-44de29234ad4 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:29.391201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18218ed1-8b7b-42bb-a3e6-44de29234ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:29.391201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18218ed1-8b7b-42bb-a3e6-44de29234ad4 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:29.394836 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 148/296] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:29.409620 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83272c7-05b7-40fc-bf46-b40eb8d4f8aa None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:29.409966 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83272c7-05b7-40fc-bf46-b40eb8d4f8aa None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:29.410156 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83272c7-05b7-40fc-bf46-b40eb8d4f8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:29.437476 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-f83272c7-05b7-40fc-bf46-b40eb8d4f8aa None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:29.438686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f83272c7-05b7-40fc-bf46-b40eb8d4f8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:29.445396 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34280111-3e68-4868-9814-bb8546b0567c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:29.446093 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34280111-3e68-4868-9814-bb8546b0567c None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:29.446369 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34280111-3e68-4868-9814-bb8546b0567c None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:29.446640 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34280111-3e68-4868-9814-bb8546b0567c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:29.447438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34280111-3e68-4868-9814-bb8546b0567c None admin] RBAC: Adding query filter params (role.id=3e3756d9482f4b9b86271eacdc19f523, scope.project.id=5a38478a0f3f4ad4b28faa3b42446f00, user.id=627120f7c56849e3930279cda5727c09) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:29.447741 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34280111-3e68-4868-9814-bb8546b0567c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:29.452793 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34280111-3e68-4868-9814-bb8546b0567c None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:29.464952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 149/297] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:58:29 2022] GET /identity/v3/role_assignments?user.id=627120f7c56849e3930279cda5727c09&scope.project.id=5a38478a0f3f4ad4b28faa3b42446f00&role.id=3e3756d9482f4b9b86271eacdc19f523 => generated 608 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:29.531471 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be8bad-a64e-4366-91df-ba4dd16b18fd None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:29.531819 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be8bad-a64e-4366-91df-ba4dd16b18fd None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:29.532093 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be8bad-a64e-4366-91df-ba4dd16b18fd None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:29.534040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 150/298] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:29.535247 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 149/299] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:29.540985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666c8ae-b228-4237-9d4d-cea7ee1b2fb1 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:29.541393 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666c8ae-b228-4237-9d4d-cea7ee1b2fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:29.541756 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666c8ae-b228-4237-9d4d-cea7ee1b2fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:29.562652 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-9666c8ae-b228-4237-9d4d-cea7ee1b2fb1 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:29.564151 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-9666c8ae-b228-4237-9d4d-cea7ee1b2fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:29.655662 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 151/300] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:29.727226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83272c7-05b7-40fc-bf46-b40eb8d4f8aa None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:29.783821 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:29.850456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83272c7-05b7-40fc-bf46-b40eb8d4f8aa None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:29.873465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9666c8ae-b228-4237-9d4d-cea7ee1b2fb1 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:29.897829 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab35e2e-be96-4d00-aaba-d588711fb084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:29.898524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab35e2e-be96-4d00-aaba-d588711fb084 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:29.898798 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab35e2e-be96-4d00-aaba-d588711fb084 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:29.899065 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab35e2e-be96-4d00-aaba-d588711fb084 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:29.901058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab35e2e-be96-4d00-aaba-d588711fb084 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:29.902578 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab35e2e-be96-4d00-aaba-d588711fb084 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:29.908575 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 150/301] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:58:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:29.909796 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab35e2e-be96-4d00-aaba-d588711fb084 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:29.928575 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:30.009563 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9666c8ae-b228-4237-9d4d-cea7ee1b2fb1 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:30.025041 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab35e2e-be96-4d00-aaba-d588711fb084 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:30.040765 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:30.046289 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b11331-33f0-445f-ab58-27973a8459ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:30.046940 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b11331-33f0-445f-ab58-27973a8459ef None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:30.047217 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b11331-33f0-445f-ab58-27973a8459ef None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:30.047485 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b11331-33f0-445f-ab58-27973a8459ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:30.061797 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b11331-33f0-445f-ab58-27973a8459ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:30.062110 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557138d-9481-4c1b-b1d7-fafe6e8ff6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:30.062731 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557138d-9481-4c1b-b1d7-fafe6e8ff6fc None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:30.063005 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557138d-9481-4c1b-b1d7-fafe6e8ff6fc None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:30.063273 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557138d-9481-4c1b-b1d7-fafe6e8ff6fc None admin] PATH_INFO: `/v3/groups` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:30.063904 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557138d-9481-4c1b-b1d7-fafe6e8ff6fc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:30.065372 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557138d-9481-4c1b-b1d7-fafe6e8ff6fc None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:30.066633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-99b11331-33f0-445f-ab58-27973a8459ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:30.066633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:30.066633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:30.066633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:30.066633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:30.066633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:30.066633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:30.066633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:30.066633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:30.066633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:30.066633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:30.066633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:30.066633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:30.066633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:30.066633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:30.066633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:30.067495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:30.067495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:30.067495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:30.067495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:30.067495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:30.067495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:30.067495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:30.067495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:30.067495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:30.067495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:30.067495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:30.067495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:30.067495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:30.067495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:30.067495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:30.067495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:30.068930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:30.068930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:30.068930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:30.068930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:30.068930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:30.068930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:30.068930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:30.068930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:30.068930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:30.068930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:30.068930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:30.068930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:30.068930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 152/302] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:58:30.068930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b11331-33f0-445f-ab58-27973a8459ef None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:30.088060 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 151/303] 10.222.0.44 () {60 vars in 1134 bytes} [Sun Sep 11 14:58:29 2022] POST /identity/v3/groups => generated 229 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 14:58:30.089498 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557138d-9481-4c1b-b1d7-fafe6e8ff6fc None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:30.213501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:30.224657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b11331-33f0-445f-ab58-27973a8459ef None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:30.231288 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557138d-9481-4c1b-b1d7-fafe6e8ff6fc None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:30.285979 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22a313-1252-416b-8315-abb7e2f3d319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:30.286212 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a37ce-5271-4f0a-8e25-553a17544826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:30.286649 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a37ce-5271-4f0a-8e25-553a17544826 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:30.286964 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a37ce-5271-4f0a-8e25-553a17544826 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:30.287231 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a37ce-5271-4f0a-8e25-553a17544826 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:30.287712 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22a313-1252-416b-8315-abb7e2f3d319 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:30.288189 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22a313-1252-416b-8315-abb7e2f3d319 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:30.288649 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22a313-1252-416b-8315-abb7e2f3d319 None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:30.289667 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22a313-1252-416b-8315-abb7e2f3d319 None admin] RBAC: Adding query filter params (name=member) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:30.289974 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22a313-1252-416b-8315-abb7e2f3d319 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:30.291495 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22a313-1252-416b-8315-abb7e2f3d319 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:30.299160 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a37ce-5271-4f0a-8e25-553a17544826 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:30.300743 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 152/304] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 14:58:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:30.304643 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-6b0a37ce-5271-4f0a-8e25-553a17544826 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:58:30.304643 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:30.304643 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:30.304643 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:30.304643 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:30.304643 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:30.304643 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:30.304643 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:30.304643 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:30.304643 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:30.304643 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:30.304643 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:30.304643 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:30.304643 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:30.304643 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:30.304643 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:30.305607 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:30.305607 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:30.305607 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:30.305607 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:30.305607 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:30.305607 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:30.305607 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:30.305607 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:30.305607 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:30.305607 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:30.305607 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:30.305607 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:30.305607 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:30.305607 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:30.305607 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:30.305607 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:30.306417 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:30.306417 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:30.306417 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:30.306417 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:30.306417 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:30.306417 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:30.306417 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:30.306417 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:30.306417 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:30.306417 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:30.306417 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:58:30.306417 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:30.307052 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22a313-1252-416b-8315-abb7e2f3d319 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:30.307122 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 153/305] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:30.314321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a37ce-5271-4f0a-8e25-553a17544826 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:30.355526 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:30.432602 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22a313-1252-416b-8315-abb7e2f3d319 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:30.440810 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a37ce-5271-4f0a-8e25-553a17544826 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:30.477461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8293e9e9-1aa0-47a0-9109-5c53cac48a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:30.478523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293e9e9-1aa0-47a0-9109-5c53cac48a1f None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:30.479254 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293e9e9-1aa0-47a0-9109-5c53cac48a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:30.479960 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293e9e9-1aa0-47a0-9109-5c53cac48a1f None admin] PATH_INFO: `/v3/users/48830eecc6a54aa6bad6b8d4e33c1684` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:30.481481 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8293e9e9-1aa0-47a0-9109-5c53cac48a1f None admin] RBAC: Authorizing `identity:get_user(user_id=48830eecc6a54aa6bad6b8d4e33c1684)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:30.483292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8293e9e9-1aa0-47a0-9109-5c53cac48a1f None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:30.483740 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46efeaa-364b-4a30-9855-56937b90997f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:30.484423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46efeaa-364b-4a30-9855-56937b90997f None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:30.484764 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46efeaa-364b-4a30-9855-56937b90997f None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:30.485101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46efeaa-364b-4a30-9855-56937b90997f None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:30.485922 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46efeaa-364b-4a30-9855-56937b90997f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:30.486306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46efeaa-364b-4a30-9855-56937b90997f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:30.486483 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 153/306] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:30 2022] GET /identity/v3/users/48830eecc6a54aa6bad6b8d4e33c1684 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:30.490397 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46efeaa-364b-4a30-9855-56937b90997f None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:30.492202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8293e9e9-1aa0-47a0-9109-5c53cac48a1f None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:30.500335 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 154/307] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 14:58:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:30.509488 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46efeaa-364b-4a30-9855-56937b90997f None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:30.530973 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:30.612527 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8293e9e9-1aa0-47a0-9109-5c53cac48a1f None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:30.615941 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:30.627669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46efeaa-364b-4a30-9855-56937b90997f None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:30.651354 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989136a-e237-43b6-835e-f24ff9adbd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:30.652566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989136a-e237-43b6-835e-f24ff9adbd0c None admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:30.653152 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989136a-e237-43b6-835e-f24ff9adbd0c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:30.653568 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989136a-e237-43b6-835e-f24ff9adbd0c None admin] PATH_INFO: `/v3/system/users/48830eecc6a54aa6bad6b8d4e33c1684/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:30.655610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c989136a-e237-43b6-835e-f24ff9adbd0c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=48830eecc6a54aa6bad6b8d4e33c1684, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:30.657276 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c989136a-e237-43b6-835e-f24ff9adbd0c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:30.668208 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f75ad-f7b4-43ac-954f-40ffb8e2872f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:30.668912 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f75ad-f7b4-43ac-954f-40ffb8e2872f None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:30.669190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f75ad-f7b4-43ac-954f-40ffb8e2872f None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:30.669510 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f75ad-f7b4-43ac-954f-40ffb8e2872f None admin] PATH_INFO: `/v3/users/442f4565a38d487eb7b0b4da070bdf0f` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:30.673245 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 154/308] 10.222.0.44 () {58 vars in 1334 bytes} [Sun Sep 11 14:58:30 2022] PUT /identity/v3/system/users/48830eecc6a54aa6bad6b8d4e33c1684/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 14:58:30.692295 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f75ad-f7b4-43ac-954f-40ffb8e2872f None admin] RBAC: Authorizing `identity:get_user(user_id=442f4565a38d487eb7b0b4da070bdf0f)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:30.694335 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f75ad-f7b4-43ac-954f-40ffb8e2872f None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:30.696504 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 155/309] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:30 2022] GET /identity/v3/users/442f4565a38d487eb7b0b4da070bdf0f => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:30.699020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91272d-ac82-4783-9cab-bf116d3fd329 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:30.699300 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91272d-ac82-4783-9cab-bf116d3fd329 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:30.699560 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91272d-ac82-4783-9cab-bf116d3fd329 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:30.701011 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 155/310] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:30.706408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f75ad-f7b4-43ac-954f-40ffb8e2872f None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:30.706919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01c7ef-da5e-4378-b7ee-1f5236d2186c None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:30.707378 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01c7ef-da5e-4378-b7ee-1f5236d2186c None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:30.709061 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01c7ef-da5e-4378-b7ee-1f5236d2186c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:30.729473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-ac01c7ef-da5e-4378-b7ee-1f5236d2186c None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:30.731387 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-ac01c7ef-da5e-4378-b7ee-1f5236d2186c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:30.806979 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:30.834557 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f75ad-f7b4-43ac-954f-40ffb8e2872f None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:30.853679 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 156/311] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:30.882113 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668d260-fbc3-4d66-ba61-b01af8a20fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:30.882725 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668d260-fbc3-4d66-ba61-b01af8a20fea None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:30.882996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668d260-fbc3-4d66-ba61-b01af8a20fea None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:30.883260 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668d260-fbc3-4d66-ba61-b01af8a20fea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:30.883981 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1668d260-fbc3-4d66-ba61-b01af8a20fea None admin] RBAC: Adding query filter params (role.id=81d58cd096394c3aad885ad2d6f60f2c, user.id=442f4565a38d487eb7b0b4da070bdf0f, scope.system=all) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:30.884282 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1668d260-fbc3-4d66-ba61-b01af8a20fea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:30.886453 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1668d260-fbc3-4d66-ba61-b01af8a20fea None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:30.891328 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 156/312] 10.222.0.44 () {56 vars in 1308 bytes} [Sun Sep 11 14:58:30 2022] GET /identity/v3/role_assignments?user.id=442f4565a38d487eb7b0b4da070bdf0f&scope.system=all&role.id=81d58cd096394c3aad885ad2d6f60f2c => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:31.124701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01c7ef-da5e-4378-b7ee-1f5236d2186c None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:31.226106 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:31.230723 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01c7ef-da5e-4378-b7ee-1f5236d2186c None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:31.267856 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83aa5a2-e64f-4e7b-9738-226ff84ae31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:31.268483 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83aa5a2-e64f-4e7b-9738-226ff84ae31d None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:31.268754 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83aa5a2-e64f-4e7b-9738-226ff84ae31d None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:31.269019 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83aa5a2-e64f-4e7b-9738-226ff84ae31d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:31.280166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83aa5a2-e64f-4e7b-9738-226ff84ae31d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:31.285017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-e83aa5a2-e64f-4e7b-9738-226ff84ae31d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:58:31.285017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:31.285017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:31.285017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:31.285017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:31.285017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:31.285017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:31.285017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:31.285017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:31.285017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:31.285017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:31.285017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:31.285017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:31.285017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:31.285017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:31.285017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:31.285918 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:31.285918 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:31.285918 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:31.285918 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:31.285918 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:31.285918 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:31.285918 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:31.285918 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:31.285918 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:31.285918 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:31.285918 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:31.285918 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:31.285918 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:31.285918 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:31.285918 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:31.285918 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:31.286686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:31.286686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:31.286686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:31.286686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:31.286686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:31.286686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:31.286686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:31.286686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:31.286686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:31.286686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:31.286686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:58:31.286686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:58:31.286686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 157/313] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:31.293819 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668d260-fbc3-4d66-ba61-b01af8a20fea None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:31.399739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668d260-fbc3-4d66-ba61-b01af8a20fea None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:31.440623 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e78d8-7c53-40ca-9461-ec9b1649e088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:31.441302 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e78d8-7c53-40ca-9461-ec9b1649e088 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:31.441831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e78d8-7c53-40ca-9461-ec9b1649e088 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:31.441831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e78d8-7c53-40ca-9461-ec9b1649e088 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:31.442515 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e78d8-7c53-40ca-9461-ec9b1649e088 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:31.442815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e78d8-7c53-40ca-9461-ec9b1649e088 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:31.444232 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e78d8-7c53-40ca-9461-ec9b1649e088 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:31.454465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 157/314] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 14:58:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:31.460847 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83aa5a2-e64f-4e7b-9738-226ff84ae31d None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:31.569586 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83aa5a2-e64f-4e7b-9738-226ff84ae31d None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:31.604795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e2216-71f7-4d9b-ab71-26c62684671c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:31.605482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e2216-71f7-4d9b-ab71-26c62684671c None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:31.605755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e2216-71f7-4d9b-ab71-26c62684671c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:31.606066 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e2216-71f7-4d9b-ab71-26c62684671c None admin] PATH_INFO: `/v3/users/5c5383e4ffaf44319af42b3b11c5c0af` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:31.607395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e2216-71f7-4d9b-ab71-26c62684671c None admin] RBAC: Authorizing `identity:get_user(user_id=5c5383e4ffaf44319af42b3b11c5c0af)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:31.608951 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e2216-71f7-4d9b-ab71-26c62684671c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:31.611032 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 158/315] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:31 2022] GET /identity/v3/users/5c5383e4ffaf44319af42b3b11c5c0af => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:31.617238 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e78d8-7c53-40ca-9461-ec9b1649e088 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:31.620980 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:31.728143 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e78d8-7c53-40ca-9461-ec9b1649e088 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:31.762768 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bdb4a5-a546-467a-85fe-656ccb15f746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:31.763383 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdb4a5-a546-467a-85fe-656ccb15f746 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:31.763651 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdb4a5-a546-467a-85fe-656ccb15f746 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:31.763913 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdb4a5-a546-467a-85fe-656ccb15f746 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:31.869214 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bdb4a5-a546-467a-85fe-656ccb15f746 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:31.870763 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bdb4a5-a546-467a-85fe-656ccb15f746 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:31.872862 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:31.980995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 158/316] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:58:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:58:31.987254 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e2216-71f7-4d9b-ab71-26c62684671c None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:32.016789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:32.099814 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e2216-71f7-4d9b-ab71-26c62684671c None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:32.141432 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87bd1a-2e96-470a-bcec-f5a8898002cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:32.142106 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87bd1a-2e96-470a-bcec-f5a8898002cb None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:32.142397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87bd1a-2e96-470a-bcec-f5a8898002cb None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:32.142663 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87bd1a-2e96-470a-bcec-f5a8898002cb None admin] PATH_INFO: `/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:32.144040 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a87bd1a-2e96-470a-bcec-f5a8898002cb None admin] RBAC: Authorizing `identity:get_project(project_id=5a38478a0f3f4ad4b28faa3b42446f00)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:32.145660 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a87bd1a-2e96-470a-bcec-f5a8898002cb None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:32.147754 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 159/317] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:31 2022] GET /identity/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:32.158448 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bdb4a5-a546-467a-85fe-656ccb15f746 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:32.278863 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bdb4a5-a546-467a-85fe-656ccb15f746 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:32.288062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:32.317236 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4236273-8e60-4fa6-9c42-bb84997f9626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:32.317875 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4236273-8e60-4fa6-9c42-bb84997f9626 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:32.318143 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4236273-8e60-4fa6-9c42-bb84997f9626 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:32.318408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4236273-8e60-4fa6-9c42-bb84997f9626 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:32.319174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4236273-8e60-4fa6-9c42-bb84997f9626 None admin] RBAC: Adding query filter params (role.id=81d58cd096394c3aad885ad2d6f60f2c, scope.project.id=5a38478a0f3f4ad4b28faa3b42446f00, user.id=5c5383e4ffaf44319af42b3b11c5c0af) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:32.319476 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4236273-8e60-4fa6-9c42-bb84997f9626 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:32.320991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4236273-8e60-4fa6-9c42-bb84997f9626 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:32.328467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 159/318] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:58:32 2022] GET /identity/v3/role_assignments?user.id=5c5383e4ffaf44319af42b3b11c5c0af&scope.project.id=5a38478a0f3f4ad4b28faa3b42446f00&role.id=81d58cd096394c3aad885ad2d6f60f2c => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:32.444455 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86315f5a-263f-4909-909e-8b16b20852f1 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:32.445031 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86315f5a-263f-4909-909e-8b16b20852f1 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:32.445031 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86315f5a-263f-4909-909e-8b16b20852f1 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:32.446737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 160/319] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:32.452634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8a7af-9518-42b8-965f-46f33646555d None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:32.453208 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8a7af-9518-42b8-965f-46f33646555d None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:32.453208 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8a7af-9518-42b8-965f-46f33646555d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:32.473225 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-8cd8a7af-9518-42b8-965f-46f33646555d None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:32.474448 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-8cd8a7af-9518-42b8-965f-46f33646555d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:32.550242 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 160/320] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:32.812962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86315f5a-263f-4909-909e-8b16b20852f1 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:32.895042 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e05a40-79f9-495d-97f2-e2f9cc2c2758 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:32.895371 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e05a40-79f9-495d-97f2-e2f9cc2c2758 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:32.895582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e05a40-79f9-495d-97f2-e2f9cc2c2758 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:32.897600 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 161/321] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:32.903475 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d1383-5c67-4f1f-b6f3-20bb4e47a45b None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:32.903806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d1383-5c67-4f1f-b6f3-20bb4e47a45b None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:32.904102 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d1383-5c67-4f1f-b6f3-20bb4e47a45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:32.920057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:32.926082 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-051d1383-5c67-4f1f-b6f3-20bb4e47a45b None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:32.927265 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-051d1383-5c67-4f1f-b6f3-20bb4e47a45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:32.928446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86315f5a-263f-4909-909e-8b16b20852f1 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:32.976962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a681d-1bca-468f-9d30-df14e60890e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:32.977858 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a681d-1bca-468f-9d30-df14e60890e0 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:32.978136 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a681d-1bca-468f-9d30-df14e60890e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:32.978548 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a681d-1bca-468f-9d30-df14e60890e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:32.979927 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a681d-1bca-468f-9d30-df14e60890e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:32.981443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a681d-1bca-468f-9d30-df14e60890e0 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:32.985189 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 161/322] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:58:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:32.989946 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a681d-1bca-468f-9d30-df14e60890e0 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:33.005220 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 162/323] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:33.095143 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a681d-1bca-468f-9d30-df14e60890e0 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:33.130054 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a008f0-3989-4cb5-b343-4ab586076c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:33.130672 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a008f0-3989-4cb5-b343-4ab586076c34 None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:33.130945 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a008f0-3989-4cb5-b343-4ab586076c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:33.131220 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a008f0-3989-4cb5-b343-4ab586076c34 None admin] PATH_INFO: `/v3/groups` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:33.131839 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a008f0-3989-4cb5-b343-4ab586076c34 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:33.133329 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a008f0-3989-4cb5-b343-4ab586076c34 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:33.150444 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 162/324] 10.222.0.44 () {60 vars in 1134 bytes} [Sun Sep 11 14:58:32 2022] POST /identity/v3/groups => generated 226 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 14:58:33.260799 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d1383-5c67-4f1f-b6f3-20bb4e47a45b None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:33.281428 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450771e-b0e1-4684-ae70-af5efbb86482 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:33.281736 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450771e-b0e1-4684-ae70-af5efbb86482 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:33.281979 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450771e-b0e1-4684-ae70-af5efbb86482 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:33.283387 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 163/325] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:33.289698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ea8fa-3a67-4cbf-bcb6-207377b9258e None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:33.290056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ea8fa-3a67-4cbf-bcb6-207377b9258e None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:33.290227 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ea8fa-3a67-4cbf-bcb6-207377b9258e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:33.298002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:33.317092 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-107ea8fa-3a67-4cbf-bcb6-207377b9258e None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:33.318977 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-107ea8fa-3a67-4cbf-bcb6-207377b9258e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:33.342277 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:33.379491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d1383-5c67-4f1f-b6f3-20bb4e47a45b None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:33.407632 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 164/326] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:33.417719 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a066444f-f74e-4f65-a6eb-3e689bfe7dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:33.418377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066444f-f74e-4f65-a6eb-3e689bfe7dfe None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:33.418733 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066444f-f74e-4f65-a6eb-3e689bfe7dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:33.419000 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066444f-f74e-4f65-a6eb-3e689bfe7dfe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:33.431634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a066444f-f74e-4f65-a6eb-3e689bfe7dfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:33.434756 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-a066444f-f74e-4f65-a6eb-3e689bfe7dfe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:33.434756 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:33.434756 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:33.434756 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:33.434756 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:33.434756 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:33.434756 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:33.434756 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:33.434756 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:33.434756 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:33.434756 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:33.434756 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:33.434756 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:33.434756 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:33.434756 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:33.434756 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:33.435657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:33.435657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:33.435657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:33.435657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:33.435657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:33.435657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:33.435657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:33.435657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:33.435657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:33.435657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:33.435657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:33.435657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:33.435657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:33.435657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:33.435657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:33.435657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:33.436439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:33.436439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:33.436439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:33.436439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:33.436439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:33.436439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:33.436439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:33.436439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:33.436439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:33.436439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:33.436439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:33.436439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:33.436439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 163/327] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:33.442019 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ea8fa-3a67-4cbf-bcb6-207377b9258e None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:33.547781 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ea8fa-3a67-4cbf-bcb6-207377b9258e None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:33.584000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4741384-28cb-4472-b130-a7d7feb3e73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:33.584626 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4741384-28cb-4472-b130-a7d7feb3e73d None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:33.584903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4741384-28cb-4472-b130-a7d7feb3e73d None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:33.585193 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4741384-28cb-4472-b130-a7d7feb3e73d None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:33.585872 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4741384-28cb-4472-b130-a7d7feb3e73d None admin] RBAC: Adding query filter params (name=member) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:33.586174 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4741384-28cb-4472-b130-a7d7feb3e73d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:33.587569 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4741384-28cb-4472-b130-a7d7feb3e73d None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:33.596959 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 165/328] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 14:58:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:33.602624 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a066444f-f74e-4f65-a6eb-3e689bfe7dfe None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:33.675130 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4741384-28cb-4472-b130-a7d7feb3e73d None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:33.699440 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:33.719089 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:33.740602 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a066444f-f74e-4f65-a6eb-3e689bfe7dfe None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:33.778516 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562a3e3-f953-474a-9c09-5f807d73b14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:33.779476 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562a3e3-f953-474a-9c09-5f807d73b14d None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:33.779751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562a3e3-f953-474a-9c09-5f807d73b14d None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:33.780167 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562a3e3-f953-474a-9c09-5f807d73b14d None admin] PATH_INFO: `/v3/users/48830eecc6a54aa6bad6b8d4e33c1684` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:33.781909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562a3e3-f953-474a-9c09-5f807d73b14d None admin] RBAC: Authorizing `identity:get_user(user_id=48830eecc6a54aa6bad6b8d4e33c1684)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:33.783364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562a3e3-f953-474a-9c09-5f807d73b14d None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:33.785278 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 164/329] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:33 2022] GET /identity/v3/users/48830eecc6a54aa6bad6b8d4e33c1684 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:33.791708 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562a3e3-f953-474a-9c09-5f807d73b14d None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:33.810051 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4741384-28cb-4472-b130-a7d7feb3e73d None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:33.849400 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7d7e2-6a6a-461f-9aa2-c7d7c3e2c0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:33.850046 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7d7e2-6a6a-461f-9aa2-c7d7c3e2c0d5 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:33.850318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7d7e2-6a6a-461f-9aa2-c7d7c3e2c0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:33.850583 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7d7e2-6a6a-461f-9aa2-c7d7c3e2c0d5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:33.870163 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7d7e2-6a6a-461f-9aa2-c7d7c3e2c0d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:33.878385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-10e7d7e2-6a6a-461f-9aa2-c7d7c3e2c0d5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:58:33.878385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:33.878385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:33.878385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:33.878385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:33.878385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:33.878385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:33.878385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:33.878385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:33.878385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:33.878385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:33.878385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:33.878385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:33.878385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:33.878385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:33.878385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:33.879328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:33.879328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:33.879328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:33.879328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:33.879328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:33.879328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:33.879328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:33.879328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:33.879328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:33.879328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:33.879328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:33.879328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:33.879328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:33.879328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:33.879328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:33.879328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:33.880087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:33.880087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:33.880087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:33.880087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:33.880087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:33.880087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:33.880087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:33.880087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:33.880087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:33.880087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:33.880087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:58:33.880087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:58:33.880880 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 166/330] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:33.887170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7d7e2-6a6a-461f-9aa2-c7d7c3e2c0d5 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:33.911741 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562a3e3-f953-474a-9c09-5f807d73b14d None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:33.953973 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1958f0e9-19ed-4d2f-b571-843efb9b8806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:33.954502 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958f0e9-19ed-4d2f-b571-843efb9b8806 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:33.954768 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958f0e9-19ed-4d2f-b571-843efb9b8806 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:33.955301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958f0e9-19ed-4d2f-b571-843efb9b8806 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:33.956151 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1958f0e9-19ed-4d2f-b571-843efb9b8806 None admin] RBAC: Adding query filter params (role.id=3e3756d9482f4b9b86271eacdc19f523, user.id=48830eecc6a54aa6bad6b8d4e33c1684, scope.system=all) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:33.956347 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1958f0e9-19ed-4d2f-b571-843efb9b8806 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:33.957810 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1958f0e9-19ed-4d2f-b571-843efb9b8806 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:33.971756 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 165/331] 10.222.0.44 () {56 vars in 1308 bytes} [Sun Sep 11 14:58:33 2022] GET /identity/v3/role_assignments?user.id=48830eecc6a54aa6bad6b8d4e33c1684&scope.system=all&role.id=3e3756d9482f4b9b86271eacdc19f523 => generated 510 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:33.990859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:34.022665 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7d7e2-6a6a-461f-9aa2-c7d7c3e2c0d5 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:34.063553 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9edeba9-93a5-4680-933c-c3fd6279509c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:34.064609 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9edeba9-93a5-4680-933c-c3fd6279509c None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:34.065255 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9edeba9-93a5-4680-933c-c3fd6279509c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:34.065724 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9edeba9-93a5-4680-933c-c3fd6279509c None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:34.066653 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9edeba9-93a5-4680-933c-c3fd6279509c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:34.067157 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9edeba9-93a5-4680-933c-c3fd6279509c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:34.068797 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9edeba9-93a5-4680-933c-c3fd6279509c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:34.079594 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 167/332] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 14:58:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:34.085388 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1958f0e9-19ed-4d2f-b571-843efb9b8806 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:34.105086 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:34.209039 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1958f0e9-19ed-4d2f-b571-843efb9b8806 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:34.246050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891dde29-6192-4d2a-ac4b-56115a29ba01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:34.246824 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891dde29-6192-4d2a-ac4b-56115a29ba01 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:34.247103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891dde29-6192-4d2a-ac4b-56115a29ba01 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:34.247376 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891dde29-6192-4d2a-ac4b-56115a29ba01 None admin] PATH_INFO: `/v3/users/442f4565a38d487eb7b0b4da070bdf0f` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:34.250517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891dde29-6192-4d2a-ac4b-56115a29ba01 None admin] RBAC: Authorizing `identity:get_user(user_id=442f4565a38d487eb7b0b4da070bdf0f)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:34.250517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891dde29-6192-4d2a-ac4b-56115a29ba01 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:34.252280 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 166/333] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:34 2022] GET /identity/v3/users/442f4565a38d487eb7b0b4da070bdf0f => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:34.260139 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9edeba9-93a5-4680-933c-c3fd6279509c None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:34.384736 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9edeba9-93a5-4680-933c-c3fd6279509c None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:34.422478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14572597-6282-401d-be98-99c2ad3438f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:34.423132 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14572597-6282-401d-be98-99c2ad3438f9 None admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:34.423402 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14572597-6282-401d-be98-99c2ad3438f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:34.423667 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14572597-6282-401d-be98-99c2ad3438f9 None admin] PATH_INFO: `/v3/system/users/442f4565a38d487eb7b0b4da070bdf0f/roles/81d58cd096394c3aad885ad2d6f60f2c` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:34.426307 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14572597-6282-401d-be98-99c2ad3438f9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=442f4565a38d487eb7b0b4da070bdf0f, role_id=81d58cd096394c3aad885ad2d6f60f2c)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:34.427686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14572597-6282-401d-be98-99c2ad3438f9 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:34.430228 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:34.442443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 168/334] 10.222.0.44 () {58 vars in 1334 bytes} [Sun Sep 11 14:58:34 2022] PUT /identity/v3/system/users/442f4565a38d487eb7b0b4da070bdf0f/roles/81d58cd096394c3aad885ad2d6f60f2c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 14:58:35.863993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f53b8-0501-409d-9166-e015526af5e8 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:35.864768 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f53b8-0501-409d-9166-e015526af5e8 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:35.864768 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f53b8-0501-409d-9166-e015526af5e8 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:35.866131 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 167/335] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:35.871710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc74821-5c92-41ee-a9df-7407b553d49a None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:35.872245 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc74821-5c92-41ee-a9df-7407b553d49a None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:35.872245 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc74821-5c92-41ee-a9df-7407b553d49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:35.898241 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-2dc74821-5c92-41ee-a9df-7407b553d49a None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:35.899452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-2dc74821-5c92-41ee-a9df-7407b553d49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:35.971977 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 169/336] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:58:36.270224 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f53b8-0501-409d-9166-e015526af5e8 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:36.370585 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:36.394412 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f53b8-0501-409d-9166-e015526af5e8 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:36.432953 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13713e47-03a3-4052-8403-4a3dab66ce55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:36.433888 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13713e47-03a3-4052-8403-4a3dab66ce55 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:36.434363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13713e47-03a3-4052-8403-4a3dab66ce55 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:36.434783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13713e47-03a3-4052-8403-4a3dab66ce55 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:36.447191 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13713e47-03a3-4052-8403-4a3dab66ce55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:36.452416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-13713e47-03a3-4052-8403-4a3dab66ce55 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:58:36.452416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:36.452416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:36.452416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:36.452416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:36.452416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:36.452416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:36.452416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:36.452416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:36.452416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:36.452416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:36.452416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:36.452416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:36.452416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:36.452416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:36.452416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:36.453586 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:36.453586 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:36.453586 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:36.453586 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:36.453586 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:36.453586 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:36.453586 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:36.453586 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:36.453586 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:36.453586 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:36.453586 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:36.453586 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:36.453586 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:36.453586 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:36.453586 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:36.453586 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:36.454606 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:36.454606 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:36.454606 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:36.454606 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:36.454606 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:36.454606 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:36.454606 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:36.454606 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:36.454606 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:36.454606 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:36.454606 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:58:36.454606 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:36.456984 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 168/337] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 14:58:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:36.463048 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc74821-5c92-41ee-a9df-7407b553d49a None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:36.488433 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894948b7-d9df-4878-9fac-84dc7cc97d5f None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:36.489084 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894948b7-d9df-4878-9fac-84dc7cc97d5f None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:36.489534 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894948b7-d9df-4878-9fac-84dc7cc97d5f None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:36.491029 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 169/338] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:58:36.497269 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ba3b3-81f6-405b-92cd-666249352423 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:36.497778 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ba3b3-81f6-405b-92cd-666249352423 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:36.498212 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ba3b3-81f6-405b-92cd-666249352423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:36.527155 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-b60ba3b3-81f6-405b-92cd-666249352423 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:36.529500 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-b60ba3b3-81f6-405b-92cd-666249352423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:36.628763 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc74821-5c92-41ee-a9df-7407b553d49a None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:36.652170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 170/339] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:58:36.671848 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:36.679615 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b1814-7fc9-41f8-a051-e851d6b27604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:36.680472 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b1814-7fc9-41f8-a051-e851d6b27604 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:36.680894 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b1814-7fc9-41f8-a051-e851d6b27604 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:36.681337 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b1814-7fc9-41f8-a051-e851d6b27604 None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:36.682168 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b1814-7fc9-41f8-a051-e851d6b27604 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:36.682615 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b1814-7fc9-41f8-a051-e851d6b27604 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:36.684203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b1814-7fc9-41f8-a051-e851d6b27604 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:36.694042 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 170/340] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:58:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:36.700112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ba3b3-81f6-405b-92cd-666249352423 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:36.728169 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6df0b7-fa6c-42b9-a8a1-2b529a8a756e None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:36.728722 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6df0b7-fa6c-42b9-a8a1-2b529a8a756e None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:36.729217 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6df0b7-fa6c-42b9-a8a1-2b529a8a756e None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:36.730700 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 171/341] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:36.745017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b75fc-0178-489a-8187-845ebf7e5768 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:36.745413 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b75fc-0178-489a-8187-845ebf7e5768 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:36.745687 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b75fc-0178-489a-8187-845ebf7e5768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:36.776925 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-671b75fc-0178-489a-8187-845ebf7e5768 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:36.778445 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-671b75fc-0178-489a-8187-845ebf7e5768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:36.789889 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:36.833761 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ba3b3-81f6-405b-92cd-666249352423 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:36.908037 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9951c91d-dcac-4ac1-85f9-14f4457d57d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:36.909045 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951c91d-dcac-4ac1-85f9-14f4457d57d2 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:36.909515 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951c91d-dcac-4ac1-85f9-14f4457d57d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:36.909989 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951c91d-dcac-4ac1-85f9-14f4457d57d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:36.936726 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 172/342] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:37.010363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9951c91d-dcac-4ac1-85f9-14f4457d57d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:37.011849 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9951c91d-dcac-4ac1-85f9-14f4457d57d2 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:37.083104 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:37.130067 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 171/343] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:58:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:58:37.135707 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b75fc-0178-489a-8187-845ebf7e5768 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:37.257089 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9951c91d-dcac-4ac1-85f9-14f4457d57d2 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:37.334928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b75fc-0178-489a-8187-845ebf7e5768 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:37.390881 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:37.406412 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375b793-34c7-486d-863b-86e6b8bbd2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:37.407489 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375b793-34c7-486d-863b-86e6b8bbd2ea None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:37.407769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375b793-34c7-486d-863b-86e6b8bbd2ea None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:37.408301 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375b793-34c7-486d-863b-86e6b8bbd2ea None admin] PATH_INFO: `/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:37.409563 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375b793-34c7-486d-863b-86e6b8bbd2ea None admin] RBAC: Authorizing `identity:get_project(project_id=b4cfdc9ec6a74df880eb9dcd5e97b98c)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:37.411060 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375b793-34c7-486d-863b-86e6b8bbd2ea None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:37.414987 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 173/344] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:37 2022] GET /identity/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:37.416406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375b793-34c7-486d-863b-86e6b8bbd2ea None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:37.493343 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9951c91d-dcac-4ac1-85f9-14f4457d57d2 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:37.541721 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8363c2f3-f512-43ed-bc93-05d70ca8156b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:37.542717 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363c2f3-f512-43ed-bc93-05d70ca8156b None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:37.543349 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363c2f3-f512-43ed-bc93-05d70ca8156b None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:37.543775 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363c2f3-f512-43ed-bc93-05d70ca8156b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:37.548610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375b793-34c7-486d-863b-86e6b8bbd2ea None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:37.561742 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8363c2f3-f512-43ed-bc93-05d70ca8156b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:37.566646 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-8363c2f3-f512-43ed-bc93-05d70ca8156b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:37.566646 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:37.566646 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:37.566646 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:37.566646 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:37.566646 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:37.566646 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:37.566646 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:37.566646 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:37.566646 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:37.566646 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:37.566646 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:37.566646 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:37.566646 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:37.566646 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:37.566646 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:37.567540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:37.567540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:37.567540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:37.567540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:37.567540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:37.567540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:37.567540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:37.567540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:37.567540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:37.567540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:37.567540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:37.567540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:37.567540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:37.567540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:37.567540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:37.567540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:37.568338 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:37.568338 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:37.568338 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:37.568338 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:37.568338 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:37.568338 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:37.568338 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:37.568338 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:37.568338 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:37.568338 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:37.568338 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:37.568338 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:37.568338 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 172/345] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:37.569002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8363c2f3-f512-43ed-bc93-05d70ca8156b None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:37.607609 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d607708-d02d-4fd0-ba73-a65985b3a87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:37.608705 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d607708-d02d-4fd0-ba73-a65985b3a87d None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:37.609267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d607708-d02d-4fd0-ba73-a65985b3a87d None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:37.609747 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d607708-d02d-4fd0-ba73-a65985b3a87d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:37.616395 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:37.625534 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d607708-d02d-4fd0-ba73-a65985b3a87d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:37.630737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-4d607708-d02d-4fd0-ba73-a65985b3a87d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:37.630737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:37.630737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:37.630737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:37.630737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:37.630737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:37.630737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:37.630737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:37.630737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:37.630737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:37.630737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:37.630737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:37.630737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:37.630737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:37.630737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:37.630737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:37.631659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:37.631659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:37.631659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:37.631659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:37.631659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:37.631659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:37.631659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:37.631659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:37.631659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:37.631659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:37.631659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:37.631659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:37.631659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:37.631659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:37.631659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:37.631659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:37.632530 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:37.632530 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:37.632530 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:37.632530 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:37.632530 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:37.632530 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:37.632530 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:37.632530 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:37.632530 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:37.632530 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:37.632530 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:37.632530 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:58:37.634982 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 174/346] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:37.637102 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657c17b-d07d-4636-81cd-9d239eac7991 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:37.637680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657c17b-d07d-4636-81cd-9d239eac7991 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:37.638156 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657c17b-d07d-4636-81cd-9d239eac7991 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:37.640189 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 175/347] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:37.641169 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657c17b-d07d-4636-81cd-9d239eac7991 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:37.696300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8363c2f3-f512-43ed-bc93-05d70ca8156b None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:37.711491 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:37.733276 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381a165-44f1-45c5-b118-736316f923d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:37.733918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381a165-44f1-45c5-b118-736316f923d5 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:37.734184 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381a165-44f1-45c5-b118-736316f923d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:37.734610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381a165-44f1-45c5-b118-736316f923d5 None admin] PATH_INFO: `/v3/groups/fcdf8e5a88f34b9d836537f330cdab21` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:37.741416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381a165-44f1-45c5-b118-736316f923d5 None admin] RBAC: Authorizing `identity:get_group(group_id=fcdf8e5a88f34b9d836537f330cdab21)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:37.743211 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381a165-44f1-45c5-b118-736316f923d5 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:37.745178 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 173/348] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 14:58:37 2022] GET /identity/v3/groups/fcdf8e5a88f34b9d836537f330cdab21 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:37.746167 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381a165-44f1-45c5-b118-736316f923d5 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:37.774655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657c17b-d07d-4636-81cd-9d239eac7991 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:37.821037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2528c-1250-4962-8d16-34663181c231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:37.821957 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2528c-1250-4962-8d16-34663181c231 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:37.822237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2528c-1250-4962-8d16-34663181c231 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:37.822616 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2528c-1250-4962-8d16-34663181c231 None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:37.823600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2528c-1250-4962-8d16-34663181c231 None admin] RBAC: Adding query filter params (name=member) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:37.823827 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2528c-1250-4962-8d16-34663181c231 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:37.825448 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2528c-1250-4962-8d16-34663181c231 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:37.836526 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 176/349] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 14:58:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:37.842304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ab886-5cf7-4b2d-9ffb-793578427857 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:37.842613 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ab886-5cf7-4b2d-9ffb-793578427857 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:37.842851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ab886-5cf7-4b2d-9ffb-793578427857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:37.863798 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-534ab886-5cf7-4b2d-9ffb-793578427857 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:37.864800 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-534ab886-5cf7-4b2d-9ffb-793578427857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:37.869514 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:37.873702 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381a165-44f1-45c5-b118-736316f923d5 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:37.911655 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03875e09-382d-4cf6-a798-a4fa936ec1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:37.913243 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03875e09-382d-4cf6-a798-a4fa936ec1ef None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:37.913748 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03875e09-382d-4cf6-a798-a4fa936ec1ef None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:37.914322 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03875e09-382d-4cf6-a798-a4fa936ec1ef None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:37.915182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03875e09-382d-4cf6-a798-a4fa936ec1ef None admin] RBAC: Adding query filter params (name=member) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:37.915731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03875e09-382d-4cf6-a798-a4fa936ec1ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:37.917475 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03875e09-382d-4cf6-a798-a4fa936ec1ef None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:37.928763 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 174/350] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 14:58:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:37.929952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03875e09-382d-4cf6-a798-a4fa936ec1ef None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:37.951683 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 177/351] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:37.952544 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534ab886-5cf7-4b2d-9ffb-793578427857 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:38.021247 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:38.045923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03875e09-382d-4cf6-a798-a4fa936ec1ef None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:38.103076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534ab886-5cf7-4b2d-9ffb-793578427857 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:38.107443 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434149d-d9d2-4ee0-a802-05154e177d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:38.108326 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434149d-d9d2-4ee0-a802-05154e177d15 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:38.108849 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434149d-d9d2-4ee0-a802-05154e177d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:38.109338 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434149d-d9d2-4ee0-a802-05154e177d15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:38.110278 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434149d-d9d2-4ee0-a802-05154e177d15 None admin] RBAC: Adding query filter params (group.id=fcdf8e5a88f34b9d836537f330cdab21, role.id=b78105372b434d86872dd04e63dce750, scope.project.id=b4cfdc9ec6a74df880eb9dcd5e97b98c) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:38.110803 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434149d-d9d2-4ee0-a802-05154e177d15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:38.112558 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434149d-d9d2-4ee0-a802-05154e177d15 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:38.119581 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 175/352] 10.222.0.44 () {56 vars in 1376 bytes} [Sun Sep 11 14:58:37 2022] GET /identity/v3/role_assignments?group.id=fcdf8e5a88f34b9d836537f330cdab21&scope.project.id=b4cfdc9ec6a74df880eb9dcd5e97b98c&role.id=b78105372b434d86872dd04e63dce750 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:38.120737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434149d-d9d2-4ee0-a802-05154e177d15 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:38.150860 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:38.155339 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e532e4a-832a-4b76-98ce-86d8e72578e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:38.156260 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e532e4a-832a-4b76-98ce-86d8e72578e0 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:38.156593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e532e4a-832a-4b76-98ce-86d8e72578e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:38.156917 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e532e4a-832a-4b76-98ce-86d8e72578e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:38.256122 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e532e4a-832a-4b76-98ce-86d8e72578e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:38.258067 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434149d-d9d2-4ee0-a802-05154e177d15 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:38.258422 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e532e4a-832a-4b76-98ce-86d8e72578e0 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:38.307051 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367dd82-9a5b-43dc-b57c-35cac637a478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:38.308124 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367dd82-9a5b-43dc-b57c-35cac637a478 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:38.311219 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367dd82-9a5b-43dc-b57c-35cac637a478 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:38.312398 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367dd82-9a5b-43dc-b57c-35cac637a478 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:38.331108 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:38.416116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 178/353] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:58:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:58:38.417689 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b1358-0c6b-4aeb-8569-1b80d91f69a1 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:38.418103 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b1358-0c6b-4aeb-8569-1b80d91f69a1 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:38.418548 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b1358-0c6b-4aeb-8569-1b80d91f69a1 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:38.419934 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 179/354] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:38.420801 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367dd82-9a5b-43dc-b57c-35cac637a478 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:38.421165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b1358-0c6b-4aeb-8569-1b80d91f69a1 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:38.424637 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367dd82-9a5b-43dc-b57c-35cac637a478 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:38.453585 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:38.556788 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 176/355] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:58:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:58:38.559277 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367dd82-9a5b-43dc-b57c-35cac637a478 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:38.565284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b1358-0c6b-4aeb-8569-1b80d91f69a1 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:38.610378 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61827808-dee4-41d2-9247-892edfabbe85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:38.611037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61827808-dee4-41d2-9247-892edfabbe85 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:38.611688 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61827808-dee4-41d2-9247-892edfabbe85 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:38.612070 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61827808-dee4-41d2-9247-892edfabbe85 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:38.627382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61827808-dee4-41d2-9247-892edfabbe85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:38.633690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-61827808-dee4-41d2-9247-892edfabbe85 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:58:38.633690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:38.633690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:38.633690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:38.633690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:38.633690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:38.633690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:38.633690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:38.633690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:38.633690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:38.633690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:38.633690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:38.633690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:38.633690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:38.633690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:38.633690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:38.634614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:38.634614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:38.634614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:38.634614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:38.634614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:38.634614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:38.634614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:38.634614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:38.634614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:38.634614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:38.634614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:38.634614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:38.634614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:38.634614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:38.634614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:38.634614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:38.635393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:38.635393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:38.635393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:38.635393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:38.635393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:38.635393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:38.635393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:38.635393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:38.635393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:38.635393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:38.635393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 14:58:38.635393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:58:38.635393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 180/356] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:38.636042 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fa122-815a-4977-a453-f4f89acb669f None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:38.636909 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fa122-815a-4977-a453-f4f89acb669f None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:38.636909 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fa122-815a-4977-a453-f4f89acb669f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:38.646657 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:38.659914 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-9e3fa122-815a-4977-a453-f4f89acb669f None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:38.661172 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-9e3fa122-815a-4977-a453-f4f89acb669f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:38.698300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367dd82-9a5b-43dc-b57c-35cac637a478 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:38.725526 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:38.742839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061ea75-26d2-420a-b2e0-6ad76d8b1f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:38.743469 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061ea75-26d2-420a-b2e0-6ad76d8b1f12 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:38.743799 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061ea75-26d2-420a-b2e0-6ad76d8b1f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:38.744065 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061ea75-26d2-420a-b2e0-6ad76d8b1f12 None admin] PATH_INFO: `/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:38.745275 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061ea75-26d2-420a-b2e0-6ad76d8b1f12 None admin] RBAC: Authorizing `identity:get_project(project_id=b4cfdc9ec6a74df880eb9dcd5e97b98c)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:38.746747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061ea75-26d2-420a-b2e0-6ad76d8b1f12 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:38.751228 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 177/357] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:38 2022] GET /identity/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:38.752107 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061ea75-26d2-420a-b2e0-6ad76d8b1f12 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:38.755133 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 181/358] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:38.757166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3fa122-815a-4977-a453-f4f89acb669f None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:38.882707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061ea75-26d2-420a-b2e0-6ad76d8b1f12 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:38.893040 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3fa122-815a-4977-a453-f4f89acb669f None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:38.924078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:38.925076 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c135d-41b4-43b6-ad90-0e11044fa98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:38.925830 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c135d-41b4-43b6-ad90-0e11044fa98d None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:38.926038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c135d-41b4-43b6-ad90-0e11044fa98d None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:38.926368 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c135d-41b4-43b6-ad90-0e11044fa98d None admin] PATH_INFO: `/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:38.929982 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c135d-41b4-43b6-ad90-0e11044fa98d None admin] RBAC: Authorizing `identity:get_project(project_id=5a38478a0f3f4ad4b28faa3b42446f00)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:38.931439 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cec5a5-990f-451e-878d-5ba4a2fa4214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:38.931931 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c135d-41b4-43b6-ad90-0e11044fa98d None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:38.932151 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cec5a5-990f-451e-878d-5ba4a2fa4214 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:38.932516 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cec5a5-990f-451e-878d-5ba4a2fa4214 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:38.932847 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cec5a5-990f-451e-878d-5ba4a2fa4214 None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:38.933662 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cec5a5-990f-451e-878d-5ba4a2fa4214 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:38.934146 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cec5a5-990f-451e-878d-5ba4a2fa4214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:38.935034 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 178/359] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:38 2022] GET /identity/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:38.935653 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cec5a5-990f-451e-878d-5ba4a2fa4214 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:38.936177 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c135d-41b4-43b6-ad90-0e11044fa98d None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:38.945583 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 182/360] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 14:58:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:38.946858 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37a331-17b7-49d6-8b8b-70b33d17b58b None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:38.947200 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37a331-17b7-49d6-8b8b-70b33d17b58b None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:38.947526 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37a331-17b7-49d6-8b8b-70b33d17b58b None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:38.952846 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 183/361] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:38.952846 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37a331-17b7-49d6-8b8b-70b33d17b58b None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:39.029799 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:39.116898 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c135d-41b4-43b6-ad90-0e11044fa98d None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:39.124951 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37a331-17b7-49d6-8b8b-70b33d17b58b None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:39.158818 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9d9f9-3eb9-471a-9266-38a9952adb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:39.159608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9d9f9-3eb9-471a-9266-38a9952adb85 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:39.159884 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9d9f9-3eb9-471a-9266-38a9952adb85 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:39.160150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9d9f9-3eb9-471a-9266-38a9952adb85 None admin] PATH_INFO: `/v3/groups/fcdf8e5a88f34b9d836537f330cdab21` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:39.161822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9d9f9-3eb9-471a-9266-38a9952adb85 None admin] RBAC: Authorizing `identity:get_group(group_id=fcdf8e5a88f34b9d836537f330cdab21)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:39.163459 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9d9f9-3eb9-471a-9266-38a9952adb85 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:39.166093 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 179/362] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 14:58:38 2022] GET /identity/v3/groups/fcdf8e5a88f34b9d836537f330cdab21 => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:39.167479 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9d9f9-3eb9-471a-9266-38a9952adb85 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:39.171224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4552928d-de68-443f-807e-55fe1cbaf627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:39.172636 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552928d-de68-443f-807e-55fe1cbaf627 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:39.172636 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552928d-de68-443f-807e-55fe1cbaf627 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:39.172636 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552928d-de68-443f-807e-55fe1cbaf627 None admin] PATH_INFO: `/v3/groups/fcdf8e5a88f34b9d836537f330cdab21` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:39.174646 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4552928d-de68-443f-807e-55fe1cbaf627 None admin] RBAC: Authorizing `identity:get_group(group_id=fcdf8e5a88f34b9d836537f330cdab21)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:39.176091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4552928d-de68-443f-807e-55fe1cbaf627 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:39.177871 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 184/363] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 14:58:38 2022] GET /identity/v3/groups/fcdf8e5a88f34b9d836537f330cdab21 => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:39.178822 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77f7cc-340f-4c33-866a-77c2bb3bbcc9 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:39.179095 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77f7cc-340f-4c33-866a-77c2bb3bbcc9 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:39.179359 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77f7cc-340f-4c33-866a-77c2bb3bbcc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:39.198282 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-7c77f7cc-340f-4c33-866a-77c2bb3bbcc9 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:39.199462 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-7c77f7cc-340f-4c33-866a-77c2bb3bbcc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:39.287658 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:39.299145 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 185/364] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:39.299825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77f7cc-340f-4c33-866a-77c2bb3bbcc9 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:39.308023 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9d9f9-3eb9-471a-9266-38a9952adb85 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:39.338025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:39.352118 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ab8b6-a565-4fda-9fc4-48663796ee15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:39.353093 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ab8b6-a565-4fda-9fc4-48663796ee15 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:39.353566 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ab8b6-a565-4fda-9fc4-48663796ee15 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:39.354002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ab8b6-a565-4fda-9fc4-48663796ee15 None admin] PATH_INFO: `/v3/users/442f4565a38d487eb7b0b4da070bdf0f` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:39.355357 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ab8b6-a565-4fda-9fc4-48663796ee15 None admin] RBAC: Authorizing `identity:get_user(user_id=442f4565a38d487eb7b0b4da070bdf0f)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:39.357073 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ab8b6-a565-4fda-9fc4-48663796ee15 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:39.358889 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 180/365] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 14:58:39 2022] GET /identity/v3/users/442f4565a38d487eb7b0b4da070bdf0f => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:39.359848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ab8b6-a565-4fda-9fc4-48663796ee15 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:39.437063 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77f7cc-340f-4c33-866a-77c2bb3bbcc9 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:39.477924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ab8b6-a565-4fda-9fc4-48663796ee15 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:39.482423 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d4b5a-9d2d-4a41-b7dd-fce75ccf7013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:39.483119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d4b5a-9d2d-4a41-b7dd-fce75ccf7013 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:39.483394 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d4b5a-9d2d-4a41-b7dd-fce75ccf7013 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:39.483660 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d4b5a-9d2d-4a41-b7dd-fce75ccf7013 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:39.499299 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d4b5a-9d2d-4a41-b7dd-fce75ccf7013 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:39.505458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-dd2d4b5a-9d2d-4a41-b7dd-fce75ccf7013 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:58:39.505458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:39.505458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:39.505458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:39.505458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:39.505458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:39.505458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:39.505458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:39.505458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:39.505458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:39.505458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:39.505458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:39.505458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:39.505458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:39.505458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:39.505458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:39.506419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:39.506419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:39.506419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:39.506419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:39.506419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:39.506419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:39.506419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:39.506419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:39.506419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:39.506419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:39.506419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:39.506419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:39.506419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:39.506419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:39.506419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:39.506419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:39.507201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:39.507201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:39.507201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:39.507201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:39.507201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:39.507201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:39.507201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:39.507201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:39.507201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:39.507201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:39.507201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:58:39.507201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:58:39.507201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 186/366] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 14:58:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:39.507908 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d4b5a-9d2d-4a41-b7dd-fce75ccf7013 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:39.531691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f704f1f-8bf4-4140-93af-771c365db1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:39.532695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f704f1f-8bf4-4140-93af-771c365db1c6 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:39.533187 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f704f1f-8bf4-4140-93af-771c365db1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:39.533697 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f704f1f-8bf4-4140-93af-771c365db1c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:39.534670 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f704f1f-8bf4-4140-93af-771c365db1c6 None admin] RBAC: Adding query filter params (group.id=fcdf8e5a88f34b9d836537f330cdab21, role.id=3e3756d9482f4b9b86271eacdc19f523, scope.project.id=b4cfdc9ec6a74df880eb9dcd5e97b98c) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:39.535338 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f704f1f-8bf4-4140-93af-771c365db1c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:39.536964 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f704f1f-8bf4-4140-93af-771c365db1c6 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:39.543473 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 181/367] 10.222.0.44 () {56 vars in 1376 bytes} [Sun Sep 11 14:58:39 2022] GET /identity/v3/role_assignments?group.id=fcdf8e5a88f34b9d836537f330cdab21&scope.project.id=b4cfdc9ec6a74df880eb9dcd5e97b98c&role.id=3e3756d9482f4b9b86271eacdc19f523 => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:39.544625 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f704f1f-8bf4-4140-93af-771c365db1c6 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:39.572661 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:39.609305 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:39.630324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d4b5a-9d2d-4a41-b7dd-fce75ccf7013 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:39.660446 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f704f1f-8bf4-4140-93af-771c365db1c6 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:39.670222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0df79-056b-47a0-a143-882664060144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:39.670558 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0df79-056b-47a0-a143-882664060144 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:39.670807 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0df79-056b-47a0-a143-882664060144 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:39.671069 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0df79-056b-47a0-a143-882664060144 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:39.671802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0df79-056b-47a0-a143-882664060144 None admin] RBAC: Adding query filter params (group.id=fcdf8e5a88f34b9d836537f330cdab21, role.id=3e3756d9482f4b9b86271eacdc19f523, scope.project.id=5a38478a0f3f4ad4b28faa3b42446f00) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:39.672102 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0df79-056b-47a0-a143-882664060144 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:39.673662 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0df79-056b-47a0-a143-882664060144 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:39.679992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 187/368] 10.222.0.44 () {56 vars in 1376 bytes} [Sun Sep 11 14:58:39 2022] GET /identity/v3/role_assignments?group.id=fcdf8e5a88f34b9d836537f330cdab21&scope.project.id=5a38478a0f3f4ad4b28faa3b42446f00&role.id=3e3756d9482f4b9b86271eacdc19f523 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:39.680833 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0df79-056b-47a0-a143-882664060144 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:39.706015 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378fcac-677e-4929-942a-3559b53c4c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:39.707060 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378fcac-677e-4929-942a-3559b53c4c46 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:39.707596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378fcac-677e-4929-942a-3559b53c4c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:39.708072 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378fcac-677e-4929-942a-3559b53c4c46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:39.709100 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e378fcac-677e-4929-942a-3559b53c4c46 None admin] RBAC: Adding query filter params (role.id=81d58cd096394c3aad885ad2d6f60f2c, user.id=442f4565a38d487eb7b0b4da070bdf0f, scope.system=all) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:39.709678 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e378fcac-677e-4929-942a-3559b53c4c46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:39.711470 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e378fcac-677e-4929-942a-3559b53c4c46 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:39.726773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 182/369] 10.222.0.44 () {56 vars in 1308 bytes} [Sun Sep 11 14:58:39 2022] GET /identity/v3/role_assignments?user.id=442f4565a38d487eb7b0b4da070bdf0f&scope.system=all&role.id=81d58cd096394c3aad885ad2d6f60f2c => generated 510 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:39.728662 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378fcac-677e-4929-942a-3559b53c4c46 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:39.816720 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0df79-056b-47a0-a143-882664060144 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:39.854923 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:39.867101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378fcac-677e-4929-942a-3559b53c4c46 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:39.909024 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:39.917370 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e944c8-1832-4bff-a309-75788194d7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:39.918038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e944c8-1832-4bff-a309-75788194d7a1 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:39.918358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e944c8-1832-4bff-a309-75788194d7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:39.918622 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e944c8-1832-4bff-a309-75788194d7a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:39.934260 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c539b6-4636-4cd7-8df8-c5f58c11059e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:39.934931 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c539b6-4636-4cd7-8df8-c5f58c11059e None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:39.935208 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c539b6-4636-4cd7-8df8-c5f58c11059e None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:39.935475 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c539b6-4636-4cd7-8df8-c5f58c11059e None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:39.936168 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c539b6-4636-4cd7-8df8-c5f58c11059e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:39.936470 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c539b6-4636-4cd7-8df8-c5f58c11059e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:39.937914 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c539b6-4636-4cd7-8df8-c5f58c11059e None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:39.941768 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e944c8-1832-4bff-a309-75788194d7a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:39.947648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-58e944c8-1832-4bff-a309-75788194d7a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:58:39.947648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:39.947648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:39.947648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:39.947648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:39.947648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:39.947648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:39.947648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:39.947648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:39.947648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:39.947648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:39.947648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:39.947648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:39.947648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:39.947648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:39.947648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:39.948581 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:39.948581 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:39.948581 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:39.948581 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:39.948581 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:39.948581 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:39.948581 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:39.948581 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:39.948581 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:39.948581 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:39.948581 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:39.948581 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:39.948581 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:39.948581 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:39.948581 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:39.948581 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:39.949407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:39.949407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:39.949407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:39.949407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:39.949407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:39.949407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:39.949407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:39.949407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:39.949407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:39.949407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:39.949407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:58:39.949407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:39.954120 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 183/370] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 14:58:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:39.962318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 188/371] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:58:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:39.963772 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e944c8-1832-4bff-a309-75788194d7a1 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:39.982530 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c539b6-4636-4cd7-8df8-c5f58c11059e None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:40.122591 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e944c8-1832-4bff-a309-75788194d7a1 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:40.157227 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c539b6-4636-4cd7-8df8-c5f58c11059e None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:40.161584 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f913c-4d12-4a29-ac85-68b92a91013f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:40.162847 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f913c-4d12-4a29-ac85-68b92a91013f None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:40.162847 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f913c-4d12-4a29-ac85-68b92a91013f None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:40.162847 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f913c-4d12-4a29-ac85-68b92a91013f None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:40.163375 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f913c-4d12-4a29-ac85-68b92a91013f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:40.163671 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f913c-4d12-4a29-ac85-68b92a91013f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:40.165053 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f913c-4d12-4a29-ac85-68b92a91013f None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:40.175747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 184/372] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:40.181911 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f913c-4d12-4a29-ac85-68b92a91013f None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:40.202117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a214128-b196-4b90-84a6-9da5e10f2082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:40.203104 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a214128-b196-4b90-84a6-9da5e10f2082 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:40.203608 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a214128-b196-4b90-84a6-9da5e10f2082 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:40.204061 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a214128-b196-4b90-84a6-9da5e10f2082 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:40.225348 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:40.274113 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:40.304522 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f913c-4d12-4a29-ac85-68b92a91013f None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:40.319277 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a214128-b196-4b90-84a6-9da5e10f2082 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:40.320897 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a214128-b196-4b90-84a6-9da5e10f2082 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:40.347427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb3d942-5469-42c3-a2df-0bcc0a8dba2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:40.348057 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3d942-5469-42c3-a2df-0bcc0a8dba2b None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:40.348329 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3d942-5469-42c3-a2df-0bcc0a8dba2b None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:40.348597 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3d942-5469-42c3-a2df-0bcc0a8dba2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:40.504830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 189/373] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:58:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:58:40.511709 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb3d942-5469-42c3-a2df-0bcc0a8dba2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:40.514012 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a214128-b196-4b90-84a6-9da5e10f2082 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:40.522714 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb3d942-5469-42c3-a2df-0bcc0a8dba2b None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:40.583892 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:40.635341 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:40.690688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 185/374] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:58:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:58:40.693055 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb3d942-5469-42c3-a2df-0bcc0a8dba2b None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:40.706510 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a214128-b196-4b90-84a6-9da5e10f2082 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:40.815209 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb3d942-5469-42c3-a2df-0bcc0a8dba2b None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:40.817604 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65e047-8e5c-4c5c-8d73-bf77f1993142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:40.818260 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65e047-8e5c-4c5c-8d73-bf77f1993142 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:40.818540 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65e047-8e5c-4c5c-8d73-bf77f1993142 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:40.818807 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65e047-8e5c-4c5c-8d73-bf77f1993142 None admin] PATH_INFO: `/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:40.820973 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd65e047-8e5c-4c5c-8d73-bf77f1993142 None admin] RBAC: Authorizing `identity:get_project(project_id=5a38478a0f3f4ad4b28faa3b42446f00)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:40.822510 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd65e047-8e5c-4c5c-8d73-bf77f1993142 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:40.826465 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 190/375] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:40 2022] GET /identity/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00 => generated 302 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:40.830091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65e047-8e5c-4c5c-8d73-bf77f1993142 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:40.882221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12607de1-637b-4d37-8803-c35c48c6cab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:40.883177 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12607de1-637b-4d37-8803-c35c48c6cab2 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:40.883646 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12607de1-637b-4d37-8803-c35c48c6cab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:40.884076 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12607de1-637b-4d37-8803-c35c48c6cab2 None admin] PATH_INFO: `/v3/projects/866dbf4792e0480eb3f18a3e97bbaa75` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:40.885982 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12607de1-637b-4d37-8803-c35c48c6cab2 None admin] RBAC: Authorizing `identity:get_project(project_id=866dbf4792e0480eb3f18a3e97bbaa75)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:40.887971 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12607de1-637b-4d37-8803-c35c48c6cab2 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:40.900278 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 186/376] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:40 2022] GET /identity/v3/projects/866dbf4792e0480eb3f18a3e97bbaa75 => generated 344 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:40.914715 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a07bbc-82e9-42c5-af13-44bd84669143 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:40.915304 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a07bbc-82e9-42c5-af13-44bd84669143 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:40.915879 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a07bbc-82e9-42c5-af13-44bd84669143 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:40.917606 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 187/377] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:40.918616 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a07bbc-82e9-42c5-af13-44bd84669143 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:40.925573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:40.999478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65e047-8e5c-4c5c-8d73-bf77f1993142 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:41.044339 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e80b23-7ab3-4d1b-8445-a68a0bba6ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:41.045088 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e80b23-7ab3-4d1b-8445-a68a0bba6ebb None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:41.045390 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e80b23-7ab3-4d1b-8445-a68a0bba6ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:41.045656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e80b23-7ab3-4d1b-8445-a68a0bba6ebb None admin] PATH_INFO: `/v3/groups/fcdf8e5a88f34b9d836537f330cdab21` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:41.047944 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e80b23-7ab3-4d1b-8445-a68a0bba6ebb None admin] RBAC: Authorizing `identity:get_group(group_id=fcdf8e5a88f34b9d836537f330cdab21)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:41.049743 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e80b23-7ab3-4d1b-8445-a68a0bba6ebb None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:41.052465 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 191/378] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 14:58:40 2022] GET /identity/v3/groups/fcdf8e5a88f34b9d836537f330cdab21 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:41.053848 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8e872-e6a5-42d5-a012-2104c40ece7e None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:41.054479 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8e872-e6a5-42d5-a012-2104c40ece7e None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:41.054758 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8e872-e6a5-42d5-a012-2104c40ece7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:41.071026 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a07bbc-82e9-42c5-af13-44bd84669143 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:41.087738 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-e6b8e872-e6a5-42d5-a012-2104c40ece7e None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:41.089006 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-e6b8e872-e6a5-42d5-a012-2104c40ece7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:41.101553 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:41.121540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40296cd1-24c3-4997-bf1e-81b186e8d938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:41.122270 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40296cd1-24c3-4997-bf1e-81b186e8d938 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:41.122546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40296cd1-24c3-4997-bf1e-81b186e8d938 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:41.122824 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40296cd1-24c3-4997-bf1e-81b186e8d938 None admin] PATH_INFO: `/v3/groups/96966871309b4d369f35ee7af410a43d` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:41.130414 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40296cd1-24c3-4997-bf1e-81b186e8d938 None admin] RBAC: Authorizing `identity:get_group(group_id=96966871309b4d369f35ee7af410a43d)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:41.132624 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40296cd1-24c3-4997-bf1e-81b186e8d938 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:41.132624 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 188/379] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 14:58:40 2022] GET /identity/v3/groups/96966871309b4d369f35ee7af410a43d => generated 229 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:41.135777 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40296cd1-24c3-4997-bf1e-81b186e8d938 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:41.181762 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 192/380] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:41.183103 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b8e872-e6a5-42d5-a012-2104c40ece7e None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:41.225936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:41.279788 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40296cd1-24c3-4997-bf1e-81b186e8d938 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:41.322202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b8e872-e6a5-42d5-a012-2104c40ece7e None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:41.326971 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fcae0-cce2-4220-a3c5-fb373b1d6e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:41.327620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fcae0-cce2-4220-a3c5-fb373b1d6e38 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:41.327934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fcae0-cce2-4220-a3c5-fb373b1d6e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:41.328241 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fcae0-cce2-4220-a3c5-fb373b1d6e38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:41.329020 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873fcae0-cce2-4220-a3c5-fb373b1d6e38 None admin] RBAC: Adding query filter params (group.id=fcdf8e5a88f34b9d836537f330cdab21, role.id=b78105372b434d86872dd04e63dce750, scope.project.id=5a38478a0f3f4ad4b28faa3b42446f00) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:41.329368 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873fcae0-cce2-4220-a3c5-fb373b1d6e38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:41.331754 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873fcae0-cce2-4220-a3c5-fb373b1d6e38 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:41.338969 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 189/381] 10.222.0.44 () {56 vars in 1376 bytes} [Sun Sep 11 14:58:41 2022] GET /identity/v3/role_assignments?group.id=fcdf8e5a88f34b9d836537f330cdab21&scope.project.id=5a38478a0f3f4ad4b28faa3b42446f00&role.id=b78105372b434d86872dd04e63dce750 => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:41.370328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312e17c-38e0-4df2-822d-8a1184c38215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:41.371400 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312e17c-38e0-4df2-822d-8a1184c38215 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:41.372081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312e17c-38e0-4df2-822d-8a1184c38215 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:41.372555 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312e17c-38e0-4df2-822d-8a1184c38215 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:41.373617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312e17c-38e0-4df2-822d-8a1184c38215 None admin] RBAC: Adding query filter params (group.id=96966871309b4d369f35ee7af410a43d, role.id=c426875a15834aa5b122e2a62267d15a, scope.project.id=866dbf4792e0480eb3f18a3e97bbaa75) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:41.374183 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312e17c-38e0-4df2-822d-8a1184c38215 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:41.375988 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312e17c-38e0-4df2-822d-8a1184c38215 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:41.384343 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 193/382] 10.222.0.44 () {56 vars in 1376 bytes} [Sun Sep 11 14:58:41 2022] GET /identity/v3/role_assignments?group.id=96966871309b4d369f35ee7af410a43d&scope.project.id=866dbf4792e0480eb3f18a3e97bbaa75&role.id=c426875a15834aa5b122e2a62267d15a => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:41.566770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fcae0-cce2-4220-a3c5-fb373b1d6e38 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:41.698575 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fcae0-cce2-4220-a3c5-fb373b1d6e38 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:41.731698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:41.736802 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd16d14-8457-4a39-9f65-9943d7492da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:41.737486 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd16d14-8457-4a39-9f65-9943d7492da8 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:41.737765 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd16d14-8457-4a39-9f65-9943d7492da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:41.738034 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd16d14-8457-4a39-9f65-9943d7492da8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:41.749578 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd16d14-8457-4a39-9f65-9943d7492da8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:41.755157 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-2cd16d14-8457-4a39-9f65-9943d7492da8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:58:41.755157 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:41.755157 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:41.755157 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:41.755157 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:41.755157 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:41.755157 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:41.755157 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:41.755157 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:41.755157 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:41.755157 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:41.755157 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:41.755157 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:41.755157 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:41.755157 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:41.755157 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:41.756103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:41.756103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:41.756103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:41.756103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:41.756103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:41.756103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:41.756103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:41.756103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:41.756103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:41.756103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:41.756103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:41.756103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:41.756103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:41.756103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:41.756103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:41.756103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:41.756876 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:41.756876 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:41.756876 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:41.756876 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:41.756876 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:41.756876 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:41.756876 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:41.756876 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:41.756876 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:41.756876 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:41.756876 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:58:41.756876 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:41.757985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 190/383] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 14:58:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:41.763659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312e17c-38e0-4df2-822d-8a1184c38215 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:41.784454 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:41.936311 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312e17c-38e0-4df2-822d-8a1184c38215 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:41.980330 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e151b-651c-408d-ae0e-095f26385c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:41.981432 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e151b-651c-408d-ae0e-095f26385c4c None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:41.981916 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e151b-651c-408d-ae0e-095f26385c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:41.982377 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e151b-651c-408d-ae0e-095f26385c4c None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:41.983283 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e151b-651c-408d-ae0e-095f26385c4c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:41.983773 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e151b-651c-408d-ae0e-095f26385c4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:41.985487 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e151b-651c-408d-ae0e-095f26385c4c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:41.997887 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 194/384] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:58:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:42.003179 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd16d14-8457-4a39-9f65-9943d7492da8 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:42.130540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd16d14-8457-4a39-9f65-9943d7492da8 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:42.165677 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb7a03-684b-4106-bcb9-1d7aa89a3946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:42.166335 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb7a03-684b-4106-bcb9-1d7aa89a3946 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:42.166612 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb7a03-684b-4106-bcb9-1d7aa89a3946 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:42.166879 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb7a03-684b-4106-bcb9-1d7aa89a3946 None admin] PATH_INFO: `/v3/groups/fcdf8e5a88f34b9d836537f330cdab21` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:42.168322 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb7a03-684b-4106-bcb9-1d7aa89a3946 None admin] RBAC: Authorizing `identity:get_group(group_id=fcdf8e5a88f34b9d836537f330cdab21)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:42.169858 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb7a03-684b-4106-bcb9-1d7aa89a3946 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:42.175416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 191/385] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 14:58:42 2022] GET /identity/v3/groups/fcdf8e5a88f34b9d836537f330cdab21 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:42.181035 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e151b-651c-408d-ae0e-095f26385c4c None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:42.295056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:42.334112 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e151b-651c-408d-ae0e-095f26385c4c None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:42.381100 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fdeb7-c1df-4c42-8cfa-2afa1f5b0c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:42.381765 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fdeb7-c1df-4c42-8cfa-2afa1f5b0c3f None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:42.382039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fdeb7-c1df-4c42-8cfa-2afa1f5b0c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:42.382300 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fdeb7-c1df-4c42-8cfa-2afa1f5b0c3f None admin] PATH_INFO: `/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:42.383443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fdeb7-c1df-4c42-8cfa-2afa1f5b0c3f None admin] RBAC: Authorizing `identity:get_project(project_id=b4cfdc9ec6a74df880eb9dcd5e97b98c)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:42.384879 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fdeb7-c1df-4c42-8cfa-2afa1f5b0c3f None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:42.387627 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 195/386] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:42 2022] GET /identity/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:42.392298 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb7a03-684b-4106-bcb9-1d7aa89a3946 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:42.517572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:42.542935 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb7a03-684b-4106-bcb9-1d7aa89a3946 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:42.600056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b75f4-9ee9-4d46-a873-0066095b7bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:42.600764 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b75f4-9ee9-4d46-a873-0066095b7bbe None admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:42.601047 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b75f4-9ee9-4d46-a873-0066095b7bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:42.601354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b75f4-9ee9-4d46-a873-0066095b7bbe None admin] PATH_INFO: `/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c/groups/fcdf8e5a88f34b9d836537f330cdab21/roles/b78105372b434d86872dd04e63dce750` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:42.603741 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b75f4-9ee9-4d46-a873-0066095b7bbe None admin] RBAC: Authorizing `identity:create_grant(project_id=b4cfdc9ec6a74df880eb9dcd5e97b98c, group_id=fcdf8e5a88f34b9d836537f330cdab21, role_id=b78105372b434d86872dd04e63dce750)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:42.605309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b75f4-9ee9-4d46-a873-0066095b7bbe None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:42.632819 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 192/387] 10.222.0.44 () {58 vars in 1442 bytes} [Sun Sep 11 14:58:42 2022] PUT /identity/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c/groups/fcdf8e5a88f34b9d836537f330cdab21/roles/b78105372b434d86872dd04e63dce750 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:58:42.711246 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a805b6f-b3f9-4411-9792-87490b5be27a None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:42.711586 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a805b6f-b3f9-4411-9792-87490b5be27a None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:42.711965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a805b6f-b3f9-4411-9792-87490b5be27a None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:42.713525 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 196/388] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:42.719176 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ba8b8-62e1-4e80-ae5f-eee8028d3ca8 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:42.719447 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ba8b8-62e1-4e80-ae5f-eee8028d3ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:42.719719 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ba8b8-62e1-4e80-ae5f-eee8028d3ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:42.728220 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32ffc5-2bd1-4abf-a6fd-ead0a3e21dae None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:42.728539 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32ffc5-2bd1-4abf-a6fd-ead0a3e21dae None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:42.728757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32ffc5-2bd1-4abf-a6fd-ead0a3e21dae None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:42.730153 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 197/389] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:42.737215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d24e7d-5796-4dba-bbfe-993101d7a1f7 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:42.737528 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d24e7d-5796-4dba-bbfe-993101d7a1f7 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:42.737752 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d24e7d-5796-4dba-bbfe-993101d7a1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:42.746947 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-480ba8b8-62e1-4e80-ae5f-eee8028d3ca8 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:42.748258 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-480ba8b8-62e1-4e80-ae5f-eee8028d3ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:42.773872 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-f7d24e7d-5796-4dba-bbfe-993101d7a1f7 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:42.775139 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f7d24e7d-5796-4dba-bbfe-993101d7a1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:42.910998 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 193/390] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:42.923636 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 198/391] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:58:43.207547 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ba8b8-62e1-4e80-ae5f-eee8028d3ca8 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:43.268603 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:43.316341 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d24e7d-5796-4dba-bbfe-993101d7a1f7 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:43.331283 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ba8b8-62e1-4e80-ae5f-eee8028d3ca8 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:43.369118 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8723bd4c-bd7a-4c06-bb63-5b12d0001ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:43.369787 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723bd4c-bd7a-4c06-bb63-5b12d0001ccb None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:43.370108 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723bd4c-bd7a-4c06-bb63-5b12d0001ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:43.370656 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723bd4c-bd7a-4c06-bb63-5b12d0001ccb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:43.382846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8723bd4c-bd7a-4c06-bb63-5b12d0001ccb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:43.388806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-8723bd4c-bd7a-4c06-bb63-5b12d0001ccb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:43.388806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:43.388806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:43.388806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:43.388806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:43.388806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:43.388806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:43.388806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:43.388806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:43.388806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:43.388806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:43.388806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:43.388806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:43.388806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:43.388806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:43.388806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:43.389731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:43.389731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:43.389731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:43.389731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:43.389731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:43.389731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:43.389731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:43.389731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:43.389731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:43.389731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:43.389731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:43.389731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:43.389731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:43.389731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:43.389731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:43.389731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:43.390517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:43.390517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:43.390517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:43.390517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:43.390517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:43.390517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:43.390517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:43.390517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:43.390517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:43.390517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:43.390517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:43.390517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:43.390517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 194/392] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:43.397467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8723bd4c-bd7a-4c06-bb63-5b12d0001ccb None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:43.459749 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:43.508810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d24e7d-5796-4dba-bbfe-993101d7a1f7 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:43.546405 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b208a-b072-45a1-8f95-73255b4b53dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:43.547054 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b208a-b072-45a1-8f95-73255b4b53dd None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:43.547327 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b208a-b072-45a1-8f95-73255b4b53dd None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:43.547592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b208a-b072-45a1-8f95-73255b4b53dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:43.559645 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b208a-b072-45a1-8f95-73255b4b53dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:43.564736 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-785b208a-b072-45a1-8f95-73255b4b53dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:43.564736 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:43.564736 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:43.564736 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:43.564736 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:43.564736 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:43.564736 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:43.564736 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:43.564736 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:43.564736 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:43.564736 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:43.564736 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:43.564736 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:43.564736 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:43.564736 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:43.564736 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:43.565656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:43.565656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:43.565656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:43.565656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:43.565656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:43.565656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:43.565656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:43.565656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:43.565656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:43.565656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:43.565656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:43.565656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:43.565656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:43.565656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:43.565656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:43.565656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:43.566452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:43.566452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:43.566452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:43.566452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:43.566452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:43.566452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:43.566452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:43.566452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:43.566452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:43.566452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:43.566452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:43.566452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:58:43.569579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 199/393] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:43.572662 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b208a-b072-45a1-8f95-73255b4b53dd None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:43.628596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8723bd4c-bd7a-4c06-bb63-5b12d0001ccb None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:43.648557 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:43.687684 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b208a-b072-45a1-8f95-73255b4b53dd None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:43.702004 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd8b84-fc89-4aee-a611-da1b74803a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:43.702649 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd8b84-fc89-4aee-a611-da1b74803a4f None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:43.702926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd8b84-fc89-4aee-a611-da1b74803a4f None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:43.703195 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd8b84-fc89-4aee-a611-da1b74803a4f None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:43.703891 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd8b84-fc89-4aee-a611-da1b74803a4f None admin] RBAC: Adding query filter params (name=member) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:43.704188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd8b84-fc89-4aee-a611-da1b74803a4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:43.705678 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd8b84-fc89-4aee-a611-da1b74803a4f None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:43.715401 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 195/394] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 14:58:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:43.721574 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd8b84-fc89-4aee-a611-da1b74803a4f None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:43.722663 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:43.731599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51b756-a267-4cda-a21f-6a6297d85357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:43.732230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51b756-a267-4cda-a21f-6a6297d85357 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:43.732551 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51b756-a267-4cda-a21f-6a6297d85357 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:43.732819 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51b756-a267-4cda-a21f-6a6297d85357 None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:43.733597 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51b756-a267-4cda-a21f-6a6297d85357 None admin] RBAC: Adding query filter params (name=member) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:43.733905 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51b756-a267-4cda-a21f-6a6297d85357 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:43.736501 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51b756-a267-4cda-a21f-6a6297d85357 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:43.753710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 200/395] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 14:58:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:43.755318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51b756-a267-4cda-a21f-6a6297d85357 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:43.896974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd8b84-fc89-4aee-a611-da1b74803a4f None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:43.965082 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51b756-a267-4cda-a21f-6a6297d85357 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:43.982559 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b169a25e-c418-448b-af70-b06945c17351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:43.983148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169a25e-c418-448b-af70-b06945c17351 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:43.983422 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169a25e-c418-448b-af70-b06945c17351 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:43.983689 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169a25e-c418-448b-af70-b06945c17351 None admin] PATH_INFO: `/v3/groups/fcdf8e5a88f34b9d836537f330cdab21` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:43.984994 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b169a25e-c418-448b-af70-b06945c17351 None admin] RBAC: Authorizing `identity:get_group(group_id=fcdf8e5a88f34b9d836537f330cdab21)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:43.986593 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b169a25e-c418-448b-af70-b06945c17351 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:43.988698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 196/396] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 14:58:43 2022] GET /identity/v3/groups/fcdf8e5a88f34b9d836537f330cdab21 => generated 226 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:44.001606 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b169a25e-c418-448b-af70-b06945c17351 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:44.040907 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111fb25-65e1-46f3-81b8-c55acc514639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:44.041872 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111fb25-65e1-46f3-81b8-c55acc514639 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:44.042292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111fb25-65e1-46f3-81b8-c55acc514639 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:44.042702 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111fb25-65e1-46f3-81b8-c55acc514639 None admin] PATH_INFO: `/v3/groups/fcdf8e5a88f34b9d836537f330cdab21` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:44.044982 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9111fb25-65e1-46f3-81b8-c55acc514639 None admin] RBAC: Authorizing `identity:get_group(group_id=fcdf8e5a88f34b9d836537f330cdab21)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:44.046657 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9111fb25-65e1-46f3-81b8-c55acc514639 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:44.051499 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 201/397] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 14:58:43 2022] GET /identity/v3/groups/fcdf8e5a88f34b9d836537f330cdab21 => generated 226 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:44.057237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111fb25-65e1-46f3-81b8-c55acc514639 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:44.096646 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:44.172280 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:44.222811 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111fb25-65e1-46f3-81b8-c55acc514639 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:44.264691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b169a25e-c418-448b-af70-b06945c17351 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:44.278031 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be2b1c6-4eb3-456d-9213-e6ab161ab94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:44.278958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be2b1c6-4eb3-456d-9213-e6ab161ab94f None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:44.279504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be2b1c6-4eb3-456d-9213-e6ab161ab94f None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:44.279970 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be2b1c6-4eb3-456d-9213-e6ab161ab94f None admin] PATH_INFO: `/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:44.281633 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be2b1c6-4eb3-456d-9213-e6ab161ab94f None admin] RBAC: Authorizing `identity:get_project(project_id=b4cfdc9ec6a74df880eb9dcd5e97b98c)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:44.283379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be2b1c6-4eb3-456d-9213-e6ab161ab94f None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:44.287946 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 202/398] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:44 2022] GET /identity/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:44.294741 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be2b1c6-4eb3-456d-9213-e6ab161ab94f None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:44.345118 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c73342-ba17-4653-a5da-e6c01474a3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:44.346219 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c73342-ba17-4653-a5da-e6c01474a3c1 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:44.346794 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c73342-ba17-4653-a5da-e6c01474a3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:44.348374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c73342-ba17-4653-a5da-e6c01474a3c1 None admin] PATH_INFO: `/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:44.350049 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c73342-ba17-4653-a5da-e6c01474a3c1 None admin] RBAC: Authorizing `identity:get_project(project_id=5a38478a0f3f4ad4b28faa3b42446f00)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:44.352547 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c73342-ba17-4653-a5da-e6c01474a3c1 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:44.357080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 197/399] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:44 2022] GET /identity/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00 => generated 302 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:44.361988 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c73342-ba17-4653-a5da-e6c01474a3c1 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:44.460488 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be2b1c6-4eb3-456d-9213-e6ab161ab94f None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:44.490339 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:44.499049 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c73342-ba17-4653-a5da-e6c01474a3c1 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:44.501779 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0020fc1d-05ac-4579-956c-5fd6875fafe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:44.502719 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020fc1d-05ac-4579-956c-5fd6875fafe9 None admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:44.503211 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020fc1d-05ac-4579-956c-5fd6875fafe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:44.503675 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020fc1d-05ac-4579-956c-5fd6875fafe9 None admin] PATH_INFO: `/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c/groups/fcdf8e5a88f34b9d836537f330cdab21/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:44.506009 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0020fc1d-05ac-4579-956c-5fd6875fafe9 None admin] RBAC: Authorizing `identity:create_grant(project_id=b4cfdc9ec6a74df880eb9dcd5e97b98c, group_id=fcdf8e5a88f34b9d836537f330cdab21, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:44.507865 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0020fc1d-05ac-4579-956c-5fd6875fafe9 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:44.508939 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:44.525642 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 203/400] 10.222.0.44 () {58 vars in 1442 bytes} [Sun Sep 11 14:58:44 2022] PUT /identity/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c/groups/fcdf8e5a88f34b9d836537f330cdab21/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 14:58:44.533978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a142945-c199-483a-8616-237c610a9b44 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:44.533978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a142945-c199-483a-8616-237c610a9b44 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:44.533978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a142945-c199-483a-8616-237c610a9b44 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:44.533978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 204/401] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:44.534974 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a9645-4026-4369-a55d-fca8fff9ee70 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:44.537318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a9645-4026-4369-a55d-fca8fff9ee70 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:44.537794 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a9645-4026-4369-a55d-fca8fff9ee70 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:44.543684 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 205/402] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:44.543684 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78b658-4ebc-418a-90e0-eacf34e589e9 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:44.543684 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78b658-4ebc-418a-90e0-eacf34e589e9 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:44.543684 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78b658-4ebc-418a-90e0-eacf34e589e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:44.552356 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9d40e-460d-44fd-8f37-81dabac1c720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:44.553373 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9d40e-460d-44fd-8f37-81dabac1c720 None admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:44.553792 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9d40e-460d-44fd-8f37-81dabac1c720 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:44.554209 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9d40e-460d-44fd-8f37-81dabac1c720 None admin] PATH_INFO: `/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00/groups/fcdf8e5a88f34b9d836537f330cdab21/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:44.556570 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9d40e-460d-44fd-8f37-81dabac1c720 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a38478a0f3f4ad4b28faa3b42446f00, group_id=fcdf8e5a88f34b9d836537f330cdab21, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:44.558339 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9d40e-460d-44fd-8f37-81dabac1c720 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:44.564408 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-1e78b658-4ebc-418a-90e0-eacf34e589e9 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:44.566753 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-1e78b658-4ebc-418a-90e0-eacf34e589e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:44.578311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 198/403] 10.222.0.44 () {58 vars in 1442 bytes} [Sun Sep 11 14:58:44 2022] PUT /identity/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00/groups/fcdf8e5a88f34b9d836537f330cdab21/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 14:58:44.580121 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b03a47-8fcd-46b7-b313-0e48b4f36b4b None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:44.580612 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b03a47-8fcd-46b7-b313-0e48b4f36b4b None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:44.581080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b03a47-8fcd-46b7-b313-0e48b4f36b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:44.604127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-e6b03a47-8fcd-46b7-b313-0e48b4f36b4b None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:44.605905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-e6b03a47-8fcd-46b7-b313-0e48b4f36b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:44.710491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 199/404] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:44.716033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 206/405] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:44.988521 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b03a47-8fcd-46b7-b313-0e48b4f36b4b None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:45.033666 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e78b658-4ebc-418a-90e0-eacf34e589e9 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:45.148835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e78b658-4ebc-418a-90e0-eacf34e589e9 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:45.167248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:45.181513 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:45.188900 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b03a47-8fcd-46b7-b313-0e48b4f36b4b None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:45.200660 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d413feb-f4a3-4fd8-aa5d-32b8114b900d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:45.201356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d413feb-f4a3-4fd8-aa5d-32b8114b900d None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:45.201632 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d413feb-f4a3-4fd8-aa5d-32b8114b900d None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:45.201900 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d413feb-f4a3-4fd8-aa5d-32b8114b900d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:45.214701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d413feb-f4a3-4fd8-aa5d-32b8114b900d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:45.219799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-2d413feb-f4a3-4fd8-aa5d-32b8114b900d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:58:45.219799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:45.219799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:45.219799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:45.219799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:45.219799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:45.219799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:45.219799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:45.219799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:45.219799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:45.219799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:45.219799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:45.219799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:45.219799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:45.219799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:45.219799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:45.220749 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:45.220749 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:45.220749 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:45.220749 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:45.220749 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:45.220749 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:45.220749 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:45.220749 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:45.220749 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:45.220749 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:45.220749 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:45.220749 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:45.220749 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:45.220749 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:45.220749 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:45.220749 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:45.222574 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:45.222574 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:45.222574 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:45.222574 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:45.222574 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:45.222574 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:45.222574 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:45.222574 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:45.222574 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:45.222574 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:45.222574 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:58:45.222574 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:58:45.222574 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 207/406] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 14:58:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:58:45.227717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d413feb-f4a3-4fd8-aa5d-32b8114b900d None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:45.267283 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac85fd-f563-48bb-a6da-e17943bce161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:45.268653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac85fd-f563-48bb-a6da-e17943bce161 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:45.269191 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac85fd-f563-48bb-a6da-e17943bce161 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:45.269700 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac85fd-f563-48bb-a6da-e17943bce161 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:45.299670 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ac85fd-f563-48bb-a6da-e17943bce161 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:45.304693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-a5ac85fd-f563-48bb-a6da-e17943bce161 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:58:45.304693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:45.304693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:45.304693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:45.304693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:45.304693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:45.304693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:45.304693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:45.304693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:45.304693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:45.304693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:45.304693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:45.304693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:45.304693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:45.304693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:45.304693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:45.305844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:45.305844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:45.305844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:45.305844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:45.305844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:45.305844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:45.305844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:45.305844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:45.305844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:45.305844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:45.305844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:45.305844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:45.305844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:45.305844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:45.305844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:45.305844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:45.306842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:45.306842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:45.306842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:45.306842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:45.306842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:45.306842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:45.306842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:45.306842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:45.306842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:45.306842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:45.306842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:58:45.306842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:45.306842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 200/407] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 14:58:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:58:45.312191 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac85fd-f563-48bb-a6da-e17943bce161 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:45.361670 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d413feb-f4a3-4fd8-aa5d-32b8114b900d None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:45.408005 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d7c67-ad3a-4bbb-9109-e1531013d2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:45.408786 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d7c67-ad3a-4bbb-9109-e1531013d2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:45.409179 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d7c67-ad3a-4bbb-9109-e1531013d2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:45.409569 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d7c67-ad3a-4bbb-9109-e1531013d2c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:45.410343 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d7c67-ad3a-4bbb-9109-e1531013d2c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:45.410746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d7c67-ad3a-4bbb-9109-e1531013d2c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:45.412257 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d7c67-ad3a-4bbb-9109-e1531013d2c1 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:45.421993 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 208/408] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:58:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:45.428301 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7216b-16fd-462d-806d-1847b68c8ef4 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:45.428633 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7216b-16fd-462d-806d-1847b68c8ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:45.428971 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7216b-16fd-462d-806d-1847b68c8ef4 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:45.430687 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 209/409] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:45.431476 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7216b-16fd-462d-806d-1847b68c8ef4 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:45.439070 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac85fd-f563-48bb-a6da-e17943bce161 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:45.482732 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c84d2-c58a-4b77-b121-1a7df3bab1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:45.484445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c84d2-c58a-4b77-b121-1a7df3bab1d9 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:45.484806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c84d2-c58a-4b77-b121-1a7df3bab1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:45.485197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c84d2-c58a-4b77-b121-1a7df3bab1d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:45.485978 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919c84d2-c58a-4b77-b121-1a7df3bab1d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:45.486347 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919c84d2-c58a-4b77-b121-1a7df3bab1d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:45.487934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919c84d2-c58a-4b77-b121-1a7df3bab1d9 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:45.491772 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:45.502423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 201/410] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:45.503875 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bdb0c1-f3b1-435f-8d22-56d364b5abd3 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:45.504223 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bdb0c1-f3b1-435f-8d22-56d364b5abd3 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:45.504562 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bdb0c1-f3b1-435f-8d22-56d364b5abd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:45.528870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-52bdb0c1-f3b1-435f-8d22-56d364b5abd3 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:45.530279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-52bdb0c1-f3b1-435f-8d22-56d364b5abd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:45.547547 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:45.576456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7216b-16fd-462d-806d-1847b68c8ef4 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:45.618072 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 202/411] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:45.619072 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bdb0c1-f3b1-435f-8d22-56d364b5abd3 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:45.622600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388c197-180f-496b-a150-47fd41ae7894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:45.623585 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388c197-180f-496b-a150-47fd41ae7894 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:45.624056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388c197-180f-496b-a150-47fd41ae7894 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:45.624537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388c197-180f-496b-a150-47fd41ae7894 None admin] PATH_INFO: `/v3/groups/fcdf8e5a88f34b9d836537f330cdab21` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:45.626015 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388c197-180f-496b-a150-47fd41ae7894 None admin] RBAC: Authorizing `identity:get_group(group_id=fcdf8e5a88f34b9d836537f330cdab21)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:45.628021 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388c197-180f-496b-a150-47fd41ae7894 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:45.630340 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 210/412] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 14:58:45 2022] GET /identity/v3/groups/fcdf8e5a88f34b9d836537f330cdab21 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:45.636641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388c197-180f-496b-a150-47fd41ae7894 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:45.798415 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bdb0c1-f3b1-435f-8d22-56d364b5abd3 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:45.833873 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388c197-180f-496b-a150-47fd41ae7894 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:45.843134 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:45.878799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa213d-6812-4909-b021-0c8114149919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:45.879439 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa213d-6812-4909-b021-0c8114149919 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:45.879714 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa213d-6812-4909-b021-0c8114149919 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:45.879979 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa213d-6812-4909-b021-0c8114149919 None admin] PATH_INFO: `/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:45.881104 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa213d-6812-4909-b021-0c8114149919 None admin] RBAC: Authorizing `identity:get_project(project_id=5a38478a0f3f4ad4b28faa3b42446f00)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:45.881233 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d2a1c-aea3-44b4-bad9-695676433457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:45.881817 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d2a1c-aea3-44b4-bad9-695676433457 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:45.882097 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d2a1c-aea3-44b4-bad9-695676433457 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:45.882364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d2a1c-aea3-44b4-bad9-695676433457 None admin] PATH_INFO: `/v3/groups/96966871309b4d369f35ee7af410a43d` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:45.882735 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa213d-6812-4909-b021-0c8114149919 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:45.884805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 211/413] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:45 2022] GET /identity/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:45.885479 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d2a1c-aea3-44b4-bad9-695676433457 None admin] RBAC: Authorizing `identity:get_group(group_id=96966871309b4d369f35ee7af410a43d)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:45.887028 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d2a1c-aea3-44b4-bad9-695676433457 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:45.888775 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 203/414] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 14:58:45 2022] GET /identity/v3/groups/96966871309b4d369f35ee7af410a43d => generated 229 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:45.893472 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa213d-6812-4909-b021-0c8114149919 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:45.896674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d2a1c-aea3-44b4-bad9-695676433457 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:45.930468 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:46.033887 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa213d-6812-4909-b021-0c8114149919 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:46.052232 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d2a1c-aea3-44b4-bad9-695676433457 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:46.089261 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa3b22-b67f-49c7-8928-12eb7f61c7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:46.090065 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa3b22-b67f-49c7-8928-12eb7f61c7ad None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:46.090404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa3b22-b67f-49c7-8928-12eb7f61c7ad None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:46.090756 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa3b22-b67f-49c7-8928-12eb7f61c7ad None admin] PATH_INFO: `/v3/projects/866dbf4792e0480eb3f18a3e97bbaa75` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:46.092060 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efa3b22-b67f-49c7-8928-12eb7f61c7ad None admin] RBAC: Authorizing `identity:get_project(project_id=866dbf4792e0480eb3f18a3e97bbaa75)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:46.093709 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efa3b22-b67f-49c7-8928-12eb7f61c7ad None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:46.100980 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 204/415] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:45 2022] GET /identity/v3/projects/866dbf4792e0480eb3f18a3e97bbaa75 => generated 344 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:46.100980 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa3b22-b67f-49c7-8928-12eb7f61c7ad None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:46.101824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7220a-55b3-4ca2-ac86-9fa106d689ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:46.102827 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7220a-55b3-4ca2-ac86-9fa106d689ec None admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:46.103312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7220a-55b3-4ca2-ac86-9fa106d689ec None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:46.103774 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7220a-55b3-4ca2-ac86-9fa106d689ec None admin] PATH_INFO: `/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00/groups/fcdf8e5a88f34b9d836537f330cdab21/roles/b78105372b434d86872dd04e63dce750` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:46.106528 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c7220a-55b3-4ca2-ac86-9fa106d689ec None admin] RBAC: Authorizing `identity:create_grant(project_id=5a38478a0f3f4ad4b28faa3b42446f00, group_id=fcdf8e5a88f34b9d836537f330cdab21, role_id=b78105372b434d86872dd04e63dce750)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:46.108356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c7220a-55b3-4ca2-ac86-9fa106d689ec None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:46.124661 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 212/416] 10.222.0.44 () {58 vars in 1442 bytes} [Sun Sep 11 14:58:45 2022] PUT /identity/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00/groups/fcdf8e5a88f34b9d836537f330cdab21/roles/b78105372b434d86872dd04e63dce750 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:58:46.126017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7220a-55b3-4ca2-ac86-9fa106d689ec None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:46.185564 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:46.207706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:46.224211 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa3b22-b67f-49c7-8928-12eb7f61c7ad None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:46.270147 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b62f4a-9922-4732-a6eb-937c4a75b7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:46.271745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b62f4a-9922-4732-a6eb-937c4a75b7f6 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:46.272222 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b62f4a-9922-4732-a6eb-937c4a75b7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:46.272686 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b62f4a-9922-4732-a6eb-937c4a75b7f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:46.286645 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b62f4a-9922-4732-a6eb-937c4a75b7f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:46.292141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-a7b62f4a-9922-4732-a6eb-937c4a75b7f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:58:46.292141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:46.292141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:46.292141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:46.292141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:46.292141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:46.292141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:46.292141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:46.292141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:46.292141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:46.292141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:46.292141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:46.292141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:46.292141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:46.292141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:46.292141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:46.293068 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:46.293068 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:46.293068 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:46.293068 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:46.293068 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:46.293068 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:46.293068 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:46.293068 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:46.293068 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:46.293068 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:46.293068 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:46.293068 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:46.293068 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:46.293068 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:46.293068 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:46.293068 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:46.294001 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:46.294001 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:46.294001 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:46.294001 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:46.294001 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:46.294001 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:46.294001 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:46.294001 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:46.294001 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:46.294001 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:46.294001 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:58:46.294001 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:46.296469 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 205/417] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 14:58:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:46.303560 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b62f4a-9922-4732-a6eb-937c4a75b7f6 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:46.357702 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7220a-55b3-4ca2-ac86-9fa106d689ec None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:46.400334 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50055fc8-d37e-4da4-96eb-489192b5e2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:46.401018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50055fc8-d37e-4da4-96eb-489192b5e2b7 None admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:46.401346 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50055fc8-d37e-4da4-96eb-489192b5e2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:46.401601 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50055fc8-d37e-4da4-96eb-489192b5e2b7 None admin] PATH_INFO: `/v3/projects/866dbf4792e0480eb3f18a3e97bbaa75/groups/96966871309b4d369f35ee7af410a43d/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:46.405595 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50055fc8-d37e-4da4-96eb-489192b5e2b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=866dbf4792e0480eb3f18a3e97bbaa75, group_id=96966871309b4d369f35ee7af410a43d, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:46.405891 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50055fc8-d37e-4da4-96eb-489192b5e2b7 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:46.424928 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b62f4a-9922-4732-a6eb-937c4a75b7f6 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:46.426220 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 213/418] 10.222.0.44 () {58 vars in 1442 bytes} [Sun Sep 11 14:58:46 2022] PUT /identity/v3/projects/866dbf4792e0480eb3f18a3e97bbaa75/groups/96966871309b4d369f35ee7af410a43d/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:58:46.472739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76136773-7547-45e0-b061-a1e10f7f7678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:46.473666 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76136773-7547-45e0-b061-a1e10f7f7678 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:46.474112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76136773-7547-45e0-b061-a1e10f7f7678 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:46.474522 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76136773-7547-45e0-b061-a1e10f7f7678 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:46.475329 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76136773-7547-45e0-b061-a1e10f7f7678 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:46.475767 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76136773-7547-45e0-b061-a1e10f7f7678 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:46.477419 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76136773-7547-45e0-b061-a1e10f7f7678 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:46.488279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 206/419] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:58:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:46.494592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50055fc8-d37e-4da4-96eb-489192b5e2b7 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:46.632289 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:46.639542 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50055fc8-d37e-4da4-96eb-489192b5e2b7 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:46.684896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eced82-f7bf-4cc4-93d4-e5aa7a8859f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:46.685581 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eced82-f7bf-4cc4-93d4-e5aa7a8859f1 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:46.685855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eced82-f7bf-4cc4-93d4-e5aa7a8859f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:46.686119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eced82-f7bf-4cc4-93d4-e5aa7a8859f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:46.766546 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eced82-f7bf-4cc4-93d4-e5aa7a8859f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:46.768072 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eced82-f7bf-4cc4-93d4-e5aa7a8859f1 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:46.923253 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:46.937437 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 214/420] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:58:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:58:46.943416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76136773-7547-45e0-b061-a1e10f7f7678 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:46.960025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:47.086758 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76136773-7547-45e0-b061-a1e10f7f7678 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:47.136621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f8806-9783-4b5a-a53d-58e8328a6946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:47.137665 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f8806-9783-4b5a-a53d-58e8328a6946 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:47.138099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f8806-9783-4b5a-a53d-58e8328a6946 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:47.138578 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f8806-9783-4b5a-a53d-58e8328a6946 None admin] PATH_INFO: `/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:47.139919 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f8806-9783-4b5a-a53d-58e8328a6946 None admin] RBAC: Authorizing `identity:get_project(project_id=b4cfdc9ec6a74df880eb9dcd5e97b98c)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:47.141560 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f8806-9783-4b5a-a53d-58e8328a6946 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:47.143718 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 207/421] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:46 2022] GET /identity/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:47.156169 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eced82-f7bf-4cc4-93d4-e5aa7a8859f1 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:47.278601 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434ab74-845e-41a6-9d0d-264a08723453 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:47.279077 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434ab74-845e-41a6-9d0d-264a08723453 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:47.279151 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434ab74-845e-41a6-9d0d-264a08723453 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:47.280775 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 208/422] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:47.285990 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f929ec-d122-4c05-9055-d5a8826659d7 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:47.286295 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f929ec-d122-4c05-9055-d5a8826659d7 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:47.286572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f929ec-d122-4c05-9055-d5a8826659d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:47.324738 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-b9f929ec-d122-4c05-9055-d5a8826659d7 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:47.327595 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-b9f929ec-d122-4c05-9055-d5a8826659d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:47.341383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eced82-f7bf-4cc4-93d4-e5aa7a8859f1 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:47.421294 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1729bee-69b6-4579-8998-11963c38b762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:47.422817 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1729bee-69b6-4579-8998-11963c38b762 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:47.423584 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1729bee-69b6-4579-8998-11963c38b762 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:47.424416 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1729bee-69b6-4579-8998-11963c38b762 None admin] PATH_INFO: `/v3/groups/fcdf8e5a88f34b9d836537f330cdab21` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:47.426563 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1729bee-69b6-4579-8998-11963c38b762 None admin] RBAC: Authorizing `identity:get_group(group_id=fcdf8e5a88f34b9d836537f330cdab21)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:47.429662 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1729bee-69b6-4579-8998-11963c38b762 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:47.433285 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 215/423] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 14:58:47 2022] GET /identity/v3/groups/fcdf8e5a88f34b9d836537f330cdab21 => generated 226 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:47.442858 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1729bee-69b6-4579-8998-11963c38b762 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:47.492782 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 209/424] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:58:47.567071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:47.633894 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1729bee-69b6-4579-8998-11963c38b762 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:47.692908 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c09ca-ddf3-4ba6-9090-6ccb4b970b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:47.693582 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c09ca-ddf3-4ba6-9090-6ccb4b970b72 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:47.693862 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c09ca-ddf3-4ba6-9090-6ccb4b970b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:47.694136 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c09ca-ddf3-4ba6-9090-6ccb4b970b72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:47.694911 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c09ca-ddf3-4ba6-9090-6ccb4b970b72 None admin] RBAC: Adding query filter params (group.id=fcdf8e5a88f34b9d836537f330cdab21, role.id=b78105372b434d86872dd04e63dce750, scope.project.id=b4cfdc9ec6a74df880eb9dcd5e97b98c) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:47.695218 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c09ca-ddf3-4ba6-9090-6ccb4b970b72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:47.696745 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c09ca-ddf3-4ba6-9090-6ccb4b970b72 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:47.708736 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 216/425] 10.222.0.44 () {56 vars in 1376 bytes} [Sun Sep 11 14:58:47 2022] GET /identity/v3/role_assignments?group.id=fcdf8e5a88f34b9d836537f330cdab21&scope.project.id=b4cfdc9ec6a74df880eb9dcd5e97b98c&role.id=b78105372b434d86872dd04e63dce750 => generated 611 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:47.820249 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d15d8-edbd-4c54-a54f-a95b65d73437 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:47.820436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d15d8-edbd-4c54-a54f-a95b65d73437 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:47.820706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d15d8-edbd-4c54-a54f-a95b65d73437 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:47.822566 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 210/426] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:47.828705 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8810551d-6bf9-4872-9dca-82a53040e2dc None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:47.829035 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8810551d-6bf9-4872-9dca-82a53040e2dc None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:47.829313 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8810551d-6bf9-4872-9dca-82a53040e2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:47.851270 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-8810551d-6bf9-4872-9dca-82a53040e2dc None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:47.852521 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-8810551d-6bf9-4872-9dca-82a53040e2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:47.887797 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d15d8-edbd-4c54-a54f-a95b65d73437 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:47.897426 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:48.004523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 217/427] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:48.066740 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d15d8-edbd-4c54-a54f-a95b65d73437 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:48.109121 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be47c1c-2180-4093-b55b-ff8517f7325c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:48.109710 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be47c1c-2180-4093-b55b-ff8517f7325c None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:48.109979 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be47c1c-2180-4093-b55b-ff8517f7325c None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:48.110310 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be47c1c-2180-4093-b55b-ff8517f7325c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:48.125962 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be47c1c-2180-4093-b55b-ff8517f7325c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:48.131106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-2be47c1c-2180-4093-b55b-ff8517f7325c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:48.131106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:48.131106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:48.131106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:48.131106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:48.131106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:48.131106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:48.131106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:48.131106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:48.131106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:48.131106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:48.131106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:48.131106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:48.131106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:48.131106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:48.131106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:48.133229 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:48.133229 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:48.133229 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:48.133229 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:48.133229 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:48.133229 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:48.133229 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:48.133229 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:48.133229 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:48.133229 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:48.133229 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:48.133229 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:48.133229 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:48.133229 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:48.133229 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:48.133229 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:48.134945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:48.134945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:48.134945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:48.134945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:48.134945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:48.134945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:48.134945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:48.134945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:48.134945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:48.134945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:48.134945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:48.134945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:48.134945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 211/428] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:58:48.143036 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8810551d-6bf9-4872-9dca-82a53040e2dc None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:48.237439 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:48.323354 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8810551d-6bf9-4872-9dca-82a53040e2dc None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:48.387751 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3e197-9847-45dc-bcf9-6d91ed52e316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:48.388756 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3e197-9847-45dc-bcf9-6d91ed52e316 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:48.389275 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3e197-9847-45dc-bcf9-6d91ed52e316 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:48.389768 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3e197-9847-45dc-bcf9-6d91ed52e316 None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:48.391043 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3e197-9847-45dc-bcf9-6d91ed52e316 None admin] RBAC: Adding query filter params (name=member) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:48.391573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3e197-9847-45dc-bcf9-6d91ed52e316 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:48.392420 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be47c1c-2180-4093-b55b-ff8517f7325c None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:48.394182 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3e197-9847-45dc-bcf9-6d91ed52e316 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:48.413215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 218/429] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 14:58:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:48.418681 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3e197-9847-45dc-bcf9-6d91ed52e316 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:48.477906 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:48.516117 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be47c1c-2180-4093-b55b-ff8517f7325c None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:48.559693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86833520-fd6d-4718-ac35-28fbedefe0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:48.560342 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86833520-fd6d-4718-ac35-28fbedefe0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:48.560615 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86833520-fd6d-4718-ac35-28fbedefe0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:48.560883 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86833520-fd6d-4718-ac35-28fbedefe0c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:48.572651 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86833520-fd6d-4718-ac35-28fbedefe0c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:48.578734 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-86833520-fd6d-4718-ac35-28fbedefe0c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:48.578734 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:48.578734 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:48.578734 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:48.578734 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:48.578734 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:48.578734 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:48.578734 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:48.578734 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:48.578734 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:48.578734 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:48.578734 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:48.578734 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:48.578734 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:48.578734 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:48.578734 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:48.580321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:48.580321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:48.580321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:48.580321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:48.580321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:48.580321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:48.580321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:48.580321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:48.580321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:48.580321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:48.580321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:48.580321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:48.580321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:48.580321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:48.580321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:48.580321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:48.582092 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:48.582092 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:48.582092 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:48.582092 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:48.582092 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:48.582092 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:48.582092 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:48.582092 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:48.582092 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:48.582092 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:48.582092 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 14:58:48.582092 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:48.582092 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 212/430] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:48.585641 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86833520-fd6d-4718-ac35-28fbedefe0c2 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:48.613997 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3e197-9847-45dc-bcf9-6d91ed52e316 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:48.635259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:48.671083 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b734f45-b461-4c4f-b2d7-0ea5360ab5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:48.671606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b734f45-b461-4c4f-b2d7-0ea5360ab5b9 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:48.671876 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b734f45-b461-4c4f-b2d7-0ea5360ab5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:48.672121 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b734f45-b461-4c4f-b2d7-0ea5360ab5b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:48.764589 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86833520-fd6d-4718-ac35-28fbedefe0c2 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:48.798575 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:48.799169 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b734f45-b461-4c4f-b2d7-0ea5360ab5b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:48.800672 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b734f45-b461-4c4f-b2d7-0ea5360ab5b9 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:48.814846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d484d-b5ac-4f61-9946-9081d809c674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:48.815473 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d484d-b5ac-4f61-9946-9081d809c674 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:48.815745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d484d-b5ac-4f61-9946-9081d809c674 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:48.816015 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d484d-b5ac-4f61-9946-9081d809c674 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:48.816692 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d484d-b5ac-4f61-9946-9081d809c674 None admin] RBAC: Adding query filter params (name=member) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:48.816995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d484d-b5ac-4f61-9946-9081d809c674 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:48.818697 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d484d-b5ac-4f61-9946-9081d809c674 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:48.828307 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 213/431] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 14:58:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:48.829652 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8051269-c5d7-4cd9-9ecb-3b98e91cbbf7 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:48.829936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8051269-c5d7-4cd9-9ecb-3b98e91cbbf7 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:48.830208 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8051269-c5d7-4cd9-9ecb-3b98e91cbbf7 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:48.831726 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 214/432] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:48.835066 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8051269-c5d7-4cd9-9ecb-3b98e91cbbf7 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:48.959066 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8051269-c5d7-4cd9-9ecb-3b98e91cbbf7 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:48.966933 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 219/433] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:58:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:58:48.969123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104c5d3-e6e6-4ebb-a13f-6889f087df7f None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:48.969623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104c5d3-e6e6-4ebb-a13f-6889f087df7f None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:48.970095 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104c5d3-e6e6-4ebb-a13f-6889f087df7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:48.999615 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-d104c5d3-e6e6-4ebb-a13f-6889f087df7f None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:49.000715 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac04609-7bc5-4f60-bf69-4a026bb9b263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:49.001375 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac04609-7bc5-4f60-bf69-4a026bb9b263 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:49.001658 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac04609-7bc5-4f60-bf69-4a026bb9b263 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:49.001754 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-d104c5d3-e6e6-4ebb-a13f-6889f087df7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:49.001931 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac04609-7bc5-4f60-bf69-4a026bb9b263 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:49.025778 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:49.086785 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:49.092881 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 220/434] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:49.094091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d104c5d3-e6e6-4ebb-a13f-6889f087df7f None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:49.115958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac04609-7bc5-4f60-bf69-4a026bb9b263 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:49.118150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac04609-7bc5-4f60-bf69-4a026bb9b263 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:49.212524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d104c5d3-e6e6-4ebb-a13f-6889f087df7f None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:49.256022 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63842a58-c2e2-48a3-b211-fbfc044d7b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:49.256703 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63842a58-c2e2-48a3-b211-fbfc044d7b72 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:49.257444 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63842a58-c2e2-48a3-b211-fbfc044d7b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:49.257717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63842a58-c2e2-48a3-b211-fbfc044d7b72 None admin] PATH_INFO: `/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:49.259160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63842a58-c2e2-48a3-b211-fbfc044d7b72 None admin] RBAC: Authorizing `identity:get_project(project_id=b4cfdc9ec6a74df880eb9dcd5e97b98c)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:49.261220 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63842a58-c2e2-48a3-b211-fbfc044d7b72 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:49.267683 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 221/435] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:49 2022] GET /identity/v3/projects/b4cfdc9ec6a74df880eb9dcd5e97b98c => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:49.269312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 215/436] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:58:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 14:58:49.274864 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63842a58-c2e2-48a3-b211-fbfc044d7b72 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:49.276921 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac04609-7bc5-4f60-bf69-4a026bb9b263 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:49.305067 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:49.386058 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:49.403680 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac04609-7bc5-4f60-bf69-4a026bb9b263 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:49.419674 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63842a58-c2e2-48a3-b211-fbfc044d7b72 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:49.451946 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8e930-63a2-4bd7-9f45-c1624977a161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:49.453545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8e930-63a2-4bd7-9f45-c1624977a161 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:49.454000 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8e930-63a2-4bd7-9f45-c1624977a161 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:49.454421 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8e930-63a2-4bd7-9f45-c1624977a161 None admin] PATH_INFO: `/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:49.456320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de8e930-63a2-4bd7-9f45-c1624977a161 None admin] RBAC: Authorizing `identity:get_project(project_id=5a38478a0f3f4ad4b28faa3b42446f00)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:49.458466 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de8e930-63a2-4bd7-9f45-c1624977a161 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:49.465731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 216/437] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:49 2022] GET /identity/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:49.467384 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8e930-63a2-4bd7-9f45-c1624977a161 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:49.469995 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3287a8-4284-4233-b582-3cd9bc84adc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:49.470724 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3287a8-4284-4233-b582-3cd9bc84adc8 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:49.471000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3287a8-4284-4233-b582-3cd9bc84adc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:49.471274 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3287a8-4284-4233-b582-3cd9bc84adc8 None admin] PATH_INFO: `/v3/groups/fcdf8e5a88f34b9d836537f330cdab21` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:49.473025 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3287a8-4284-4233-b582-3cd9bc84adc8 None admin] RBAC: Authorizing `identity:get_group(group_id=fcdf8e5a88f34b9d836537f330cdab21)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:49.475576 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3287a8-4284-4233-b582-3cd9bc84adc8 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:49.481200 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 222/438] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 14:58:49 2022] GET /identity/v3/groups/fcdf8e5a88f34b9d836537f330cdab21 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:49.485242 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3287a8-4284-4233-b582-3cd9bc84adc8 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:49.588386 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:49.600014 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8e930-63a2-4bd7-9f45-c1624977a161 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:49.613175 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3287a8-4284-4233-b582-3cd9bc84adc8 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:49.641274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402998a8-9ab6-476f-badc-483593253d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:49.642219 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402998a8-9ab6-476f-badc-483593253d29 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:49.642510 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402998a8-9ab6-476f-badc-483593253d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:49.642777 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402998a8-9ab6-476f-badc-483593253d29 None admin] PATH_INFO: `/v3/groups/fcdf8e5a88f34b9d836537f330cdab21` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:49.644279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402998a8-9ab6-476f-badc-483593253d29 None admin] RBAC: Authorizing `identity:get_group(group_id=fcdf8e5a88f34b9d836537f330cdab21)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:49.646544 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402998a8-9ab6-476f-badc-483593253d29 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:49.656455 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 217/439] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 14:58:49 2022] GET /identity/v3/groups/fcdf8e5a88f34b9d836537f330cdab21 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:49.656455 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402998a8-9ab6-476f-badc-483593253d29 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:49.664866 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7b2754-709f-4ad8-8f35-c8bfe574a3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:49.665728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b2754-709f-4ad8-8f35-c8bfe574a3b4 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:49.666032 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b2754-709f-4ad8-8f35-c8bfe574a3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:49.666289 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b2754-709f-4ad8-8f35-c8bfe574a3b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:49.667458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7b2754-709f-4ad8-8f35-c8bfe574a3b4 None admin] RBAC: Adding query filter params (group.id=fcdf8e5a88f34b9d836537f330cdab21, role.id=3e3756d9482f4b9b86271eacdc19f523, scope.project.id=b4cfdc9ec6a74df880eb9dcd5e97b98c) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:49.667964 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7b2754-709f-4ad8-8f35-c8bfe574a3b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:49.669626 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7b2754-709f-4ad8-8f35-c8bfe574a3b4 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:49.677623 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:49.678214 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 223/440] 10.222.0.44 () {56 vars in 1376 bytes} [Sun Sep 11 14:58:49 2022] GET /identity/v3/role_assignments?group.id=fcdf8e5a88f34b9d836537f330cdab21&scope.project.id=b4cfdc9ec6a74df880eb9dcd5e97b98c&role.id=3e3756d9482f4b9b86271eacdc19f523 => generated 611 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:49.679087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7b2754-709f-4ad8-8f35-c8bfe574a3b4 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:49.796831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402998a8-9ab6-476f-badc-483593253d29 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:49.842997 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca55b0-bd72-47eb-aa1f-bd0c06698266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:49.843661 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca55b0-bd72-47eb-aa1f-bd0c06698266 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:49.843937 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca55b0-bd72-47eb-aa1f-bd0c06698266 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:49.844207 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca55b0-bd72-47eb-aa1f-bd0c06698266 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:49.860717 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca55b0-bd72-47eb-aa1f-bd0c06698266 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:49.866136 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-36ca55b0-bd72-47eb-aa1f-bd0c06698266 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:58:49.866136 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:49.866136 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:49.866136 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:49.866136 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:49.866136 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:49.866136 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:49.866136 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:49.866136 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:49.866136 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:49.866136 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:49.866136 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:49.866136 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:49.866136 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:49.866136 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:49.866136 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:49.874013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:49.874013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:49.874013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:49.874013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:49.874013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:49.874013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:49.874013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:49.874013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:49.874013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:49.874013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:49.874013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:49.874013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:49.874013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:49.874013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:49.874013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:49.874013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:49.875552 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:49.875552 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:49.875552 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:49.875552 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:49.875552 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:49.875552 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:49.875552 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:49.875552 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:49.875552 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:49.875552 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:49.875552 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:58:49.875552 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:49.875552 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 218/441] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 14:58:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:49.883122 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca55b0-bd72-47eb-aa1f-bd0c06698266 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:49.886060 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7b2754-709f-4ad8-8f35-c8bfe574a3b4 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:49.949497 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73786c6-2e44-452f-8acc-aa805fa87d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:49.950532 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73786c6-2e44-452f-8acc-aa805fa87d1b None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:49.951033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73786c6-2e44-452f-8acc-aa805fa87d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:49.951557 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73786c6-2e44-452f-8acc-aa805fa87d1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:49.952822 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73786c6-2e44-452f-8acc-aa805fa87d1b None admin] RBAC: Adding query filter params (group.id=fcdf8e5a88f34b9d836537f330cdab21, role.id=3e3756d9482f4b9b86271eacdc19f523, scope.project.id=5a38478a0f3f4ad4b28faa3b42446f00) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:49.953393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73786c6-2e44-452f-8acc-aa805fa87d1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:49.956277 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73786c6-2e44-452f-8acc-aa805fa87d1b None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:49.965518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 224/442] 10.222.0.44 () {56 vars in 1376 bytes} [Sun Sep 11 14:58:49 2022] GET /identity/v3/role_assignments?group.id=fcdf8e5a88f34b9d836537f330cdab21&scope.project.id=5a38478a0f3f4ad4b28faa3b42446f00&role.id=3e3756d9482f4b9b86271eacdc19f523 => generated 611 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:49.967222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eccb430-c08e-47ac-9f1c-16d3f56c36a3 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:49.967735 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eccb430-c08e-47ac-9f1c-16d3f56c36a3 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:49.968248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eccb430-c08e-47ac-9f1c-16d3f56c36a3 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:49.969865 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 225/443] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:49.979086 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99cdd1d-6d39-4697-a55d-1fe433aa927f None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:49.979380 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99cdd1d-6d39-4697-a55d-1fe433aa927f None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:49.979727 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99cdd1d-6d39-4697-a55d-1fe433aa927f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:50.006266 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-c99cdd1d-6d39-4697-a55d-1fe433aa927f None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:50.007534 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c99cdd1d-6d39-4697-a55d-1fe433aa927f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:50.019226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:50.053029 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:50.075579 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca55b0-bd72-47eb-aa1f-bd0c06698266 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:50.151980 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb54ac7-291b-4da6-9182-66ee548ac7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:50.153393 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb54ac7-291b-4da6-9182-66ee548ac7bd None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:50.153675 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb54ac7-291b-4da6-9182-66ee548ac7bd None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:50.153946 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb54ac7-291b-4da6-9182-66ee548ac7bd None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:50.154847 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb54ac7-291b-4da6-9182-66ee548ac7bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:50.155155 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb54ac7-291b-4da6-9182-66ee548ac7bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:50.156923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb54ac7-291b-4da6-9182-66ee548ac7bd None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:50.162981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 226/444] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:50.170682 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 219/445] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:58:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:50.176980 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99cdd1d-6d39-4697-a55d-1fe433aa927f None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:50.316067 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99cdd1d-6d39-4697-a55d-1fe433aa927f None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:50.357007 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd038b9-f1b8-44ac-a826-93c1c2269134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:50.357688 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd038b9-f1b8-44ac-a826-93c1c2269134 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:50.357956 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd038b9-f1b8-44ac-a826-93c1c2269134 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:50.358236 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd038b9-f1b8-44ac-a826-93c1c2269134 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:50.375259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:50.458437 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd038b9-f1b8-44ac-a826-93c1c2269134 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:50.461054 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd038b9-f1b8-44ac-a826-93c1c2269134 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:50.607825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 227/446] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:58:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:58:50.613447 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb54ac7-291b-4da6-9182-66ee548ac7bd None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:50.669419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd038b9-f1b8-44ac-a826-93c1c2269134 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:50.724545 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:50.748920 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb54ac7-291b-4da6-9182-66ee548ac7bd None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:50.798256 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d621e90-5112-42b4-99f7-8705dab130ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:50.799053 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d621e90-5112-42b4-99f7-8705dab130ff None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:50.799398 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d621e90-5112-42b4-99f7-8705dab130ff None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:50.799734 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d621e90-5112-42b4-99f7-8705dab130ff None admin] PATH_INFO: `/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:50.801099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d621e90-5112-42b4-99f7-8705dab130ff None admin] RBAC: Authorizing `identity:get_project(project_id=5a38478a0f3f4ad4b28faa3b42446f00)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:50.803151 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d621e90-5112-42b4-99f7-8705dab130ff None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:50.804591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd038b9-f1b8-44ac-a826-93c1c2269134 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:50.805103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 220/447] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:50 2022] GET /identity/v3/projects/5a38478a0f3f4ad4b28faa3b42446f00 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:50.812348 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d621e90-5112-42b4-99f7-8705dab130ff None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:50.832436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:50.853637 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd5177-e074-4d28-b293-39c3472e8e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:50.854593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd5177-e074-4d28-b293-39c3472e8e5a None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:50.854989 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd5177-e074-4d28-b293-39c3472e8e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:50.855417 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd5177-e074-4d28-b293-39c3472e8e5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:50.872600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bd5177-e074-4d28-b293-39c3472e8e5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:50.880818 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-23bd5177-e074-4d28-b293-39c3472e8e5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:58:50.880818 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:50.880818 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:50.880818 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:50.880818 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:50.880818 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:50.880818 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:50.880818 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:50.880818 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:50.880818 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:50.880818 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:50.880818 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:50.880818 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:50.880818 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:50.880818 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:50.880818 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:50.882058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:50.882058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:50.882058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:50.882058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:50.882058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:50.882058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:50.882058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:50.882058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:50.882058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:50.882058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:50.882058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:50.882058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:50.882058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:50.882058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:50.882058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:50.882058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:50.883447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:50.883447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:50.883447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:50.883447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:50.883447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:50.883447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:50.883447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:50.883447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:50.883447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:50.883447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:50.883447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:58:50.883447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:58:50.883447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 228/448] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 14:58:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:58:50.891839 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd5177-e074-4d28-b293-39c3472e8e5a None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:50.953688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d621e90-5112-42b4-99f7-8705dab130ff None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:51.000971 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb9a29-d25b-4f1a-9c61-6f92d01fb4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:51.001899 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb9a29-d25b-4f1a-9c61-6f92d01fb4cc None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:51.002340 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb9a29-d25b-4f1a-9c61-6f92d01fb4cc None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:51.002778 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb9a29-d25b-4f1a-9c61-6f92d01fb4cc None admin] PATH_INFO: `/v3/groups/fcdf8e5a88f34b9d836537f330cdab21` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:51.004514 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb9a29-d25b-4f1a-9c61-6f92d01fb4cc None admin] RBAC: Authorizing `identity:get_group(group_id=fcdf8e5a88f34b9d836537f330cdab21)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:51.006990 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb9a29-d25b-4f1a-9c61-6f92d01fb4cc None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:51.009674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 221/449] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 14:58:50 2022] GET /identity/v3/groups/fcdf8e5a88f34b9d836537f330cdab21 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:51.018981 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb9a29-d25b-4f1a-9c61-6f92d01fb4cc None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:51.029286 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd5177-e074-4d28-b293-39c3472e8e5a None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:51.031396 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:51.076511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec6c5f-503f-46c6-904b-c58aa8278c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:51.077351 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec6c5f-503f-46c6-904b-c58aa8278c0c None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:51.077770 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec6c5f-503f-46c6-904b-c58aa8278c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:51.078173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec6c5f-503f-46c6-904b-c58aa8278c0c None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:51.080608 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec6c5f-503f-46c6-904b-c58aa8278c0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:51.080608 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec6c5f-503f-46c6-904b-c58aa8278c0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:51.081539 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec6c5f-503f-46c6-904b-c58aa8278c0c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:51.091919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 229/450] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:58:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:51.099002 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec6c5f-503f-46c6-904b-c58aa8278c0c None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:51.125564 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:51.153011 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb9a29-d25b-4f1a-9c61-6f92d01fb4cc None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:51.195073 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a8ac1-64ad-47ef-a26e-eaa73fbbd3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:51.195795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a8ac1-64ad-47ef-a26e-eaa73fbbd3b4 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:51.196135 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a8ac1-64ad-47ef-a26e-eaa73fbbd3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:51.196468 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a8ac1-64ad-47ef-a26e-eaa73fbbd3b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:51.197430 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a8ac1-64ad-47ef-a26e-eaa73fbbd3b4 None admin] RBAC: Adding query filter params (group.id=fcdf8e5a88f34b9d836537f330cdab21, role.id=b78105372b434d86872dd04e63dce750, scope.project.id=5a38478a0f3f4ad4b28faa3b42446f00) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:51.197895 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a8ac1-64ad-47ef-a26e-eaa73fbbd3b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:51.199672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a8ac1-64ad-47ef-a26e-eaa73fbbd3b4 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:51.209427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 222/451] 10.222.0.44 () {56 vars in 1376 bytes} [Sun Sep 11 14:58:51 2022] GET /identity/v3/role_assignments?group.id=fcdf8e5a88f34b9d836537f330cdab21&scope.project.id=5a38478a0f3f4ad4b28faa3b42446f00&role.id=b78105372b434d86872dd04e63dce750 => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:51.229057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec6c5f-503f-46c6-904b-c58aa8278c0c None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:51.280602 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c84045b-05ec-4cdb-a0c5-7f82e20f25c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:51.281545 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84045b-05ec-4cdb-a0c5-7f82e20f25c4 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:51.281957 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84045b-05ec-4cdb-a0c5-7f82e20f25c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:51.282357 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84045b-05ec-4cdb-a0c5-7f82e20f25c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:51.334757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:51.385622 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c84045b-05ec-4cdb-a0c5-7f82e20f25c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:51.387234 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c84045b-05ec-4cdb-a0c5-7f82e20f25c4 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:51.511475 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 230/452] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:58:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 14:58:51.521474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a8ac1-64ad-47ef-a26e-eaa73fbbd3b4 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:51.668335 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a8ac1-64ad-47ef-a26e-eaa73fbbd3b4 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:51.706381 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e9222-4a3a-4422-b132-9e404b2e04fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:51.707020 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e9222-4a3a-4422-b132-9e404b2e04fc None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:51.707296 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e9222-4a3a-4422-b132-9e404b2e04fc None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:51.707562 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e9222-4a3a-4422-b132-9e404b2e04fc None admin] PATH_INFO: `/v3/projects/866dbf4792e0480eb3f18a3e97bbaa75` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:51.708689 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e9222-4a3a-4422-b132-9e404b2e04fc None admin] RBAC: Authorizing `identity:get_project(project_id=866dbf4792e0480eb3f18a3e97bbaa75)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:51.710177 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e9222-4a3a-4422-b132-9e404b2e04fc None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:51.711838 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 223/453] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 14:58:51 2022] GET /identity/v3/projects/866dbf4792e0480eb3f18a3e97bbaa75 => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:51.717799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c84045b-05ec-4cdb-a0c5-7f82e20f25c4 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:51.783345 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:51.826471 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c84045b-05ec-4cdb-a0c5-7f82e20f25c4 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:51.865928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90908043-bfb6-42b5-83f5-051fba6f348c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:51.866575 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90908043-bfb6-42b5-83f5-051fba6f348c None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:51.866850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90908043-bfb6-42b5-83f5-051fba6f348c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:51.867119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90908043-bfb6-42b5-83f5-051fba6f348c None admin] PATH_INFO: `/v3/groups/96966871309b4d369f35ee7af410a43d` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:51.868396 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90908043-bfb6-42b5-83f5-051fba6f348c None admin] RBAC: Authorizing `identity:get_group(group_id=96966871309b4d369f35ee7af410a43d)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:51.869960 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90908043-bfb6-42b5-83f5-051fba6f348c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:51.871751 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 231/454] 10.222.0.44 () {56 vars in 1180 bytes} [Sun Sep 11 14:58:51 2022] GET /identity/v3/groups/96966871309b4d369f35ee7af410a43d => generated 229 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:51.878088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e9222-4a3a-4422-b132-9e404b2e04fc None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:51.895621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:52.000515 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e9222-4a3a-4422-b132-9e404b2e04fc None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:52.044103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f5eda9-39b5-43e5-ba27-8798d4c920da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:52.044730 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5eda9-39b5-43e5-ba27-8798d4c920da None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:52.044988 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5eda9-39b5-43e5-ba27-8798d4c920da None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:52.045291 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5eda9-39b5-43e5-ba27-8798d4c920da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:52.046038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f5eda9-39b5-43e5-ba27-8798d4c920da None admin] RBAC: Adding query filter params (group.id=96966871309b4d369f35ee7af410a43d, role.id=c426875a15834aa5b122e2a62267d15a, scope.project.id=866dbf4792e0480eb3f18a3e97bbaa75) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:52.046343 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f5eda9-39b5-43e5-ba27-8798d4c920da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:52.047840 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f5eda9-39b5-43e5-ba27-8798d4c920da None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:52.054872 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 224/455] 10.222.0.44 () {56 vars in 1376 bytes} [Sun Sep 11 14:58:51 2022] GET /identity/v3/role_assignments?group.id=96966871309b4d369f35ee7af410a43d&scope.project.id=866dbf4792e0480eb3f18a3e97bbaa75&role.id=c426875a15834aa5b122e2a62267d15a => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:55.052126 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcda586-873b-4c89-874e-d3df7d67115c None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:55.052651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcda586-873b-4c89-874e-d3df7d67115c None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:55.052759 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcda586-873b-4c89-874e-d3df7d67115c None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:55.054076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 232/456] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:55.079466 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73892254-fe9e-4e2f-a41b-7efff1d1694c None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:55.079993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73892254-fe9e-4e2f-a41b-7efff1d1694c None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:55.080096 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73892254-fe9e-4e2f-a41b-7efff1d1694c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:55.110249 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-73892254-fe9e-4e2f-a41b-7efff1d1694c None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:55.111727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-73892254-fe9e-4e2f-a41b-7efff1d1694c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:55.114580 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeeef7a-6d46-4704-8cf9-f3b084749d0b None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:55.116055 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeeef7a-6d46-4704-8cf9-f3b084749d0b None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:55.116542 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeeef7a-6d46-4704-8cf9-f3b084749d0b None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:55.118436 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 233/457] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:58:55.125674 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1307a-9e16-4b7b-9710-d4a867983bbe None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:55.125949 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1307a-9e16-4b7b-9710-d4a867983bbe None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:55.126219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1307a-9e16-4b7b-9710-d4a867983bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:55.158642 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-34a1307a-9e16-4b7b-9710-d4a867983bbe None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:55.159853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-34a1307a-9e16-4b7b-9710-d4a867983bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:55.264995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 225/458] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:55.283653 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 234/459] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:55.481286 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049fb8f-e4b2-42a1-a545-aae0dc4f7974 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:55.481941 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049fb8f-e4b2-42a1-a545-aae0dc4f7974 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:55.482429 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049fb8f-e4b2-42a1-a545-aae0dc4f7974 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:55.484019 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 226/460] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:55.489689 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7739966f-da2a-471c-86b4-3164186f79c6 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:55.490311 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7739966f-da2a-471c-86b4-3164186f79c6 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:55.490879 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7739966f-da2a-471c-86b4-3164186f79c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:55.518305 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-7739966f-da2a-471c-86b4-3164186f79c6 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:55.520057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-7739966f-da2a-471c-86b4-3164186f79c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:55.549572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049fb8f-e4b2-42a1-a545-aae0dc4f7974 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:55.567530 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:55.631684 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:55.656772 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 235/461] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:55.703490 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049fb8f-e4b2-42a1-a545-aae0dc4f7974 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:55.721606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7739966f-da2a-471c-86b4-3164186f79c6 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:55.741850 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f8a56-1c26-4abd-bc4e-ad561376bee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:55.742922 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f8a56-1c26-4abd-bc4e-ad561376bee8 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:55.743398 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f8a56-1c26-4abd-bc4e-ad561376bee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:55.743877 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f8a56-1c26-4abd-bc4e-ad561376bee8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:55.757117 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f8a56-1c26-4abd-bc4e-ad561376bee8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:55.759065 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f8a56-1c26-4abd-bc4e-ad561376bee8 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:55.775309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-699f8a56-1c26-4abd-bc4e-ad561376bee8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:58:55.775309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:55.775309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:55.775309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:55.775309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:55.775309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:55.775309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:55.775309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:55.775309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:55.775309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:55.775309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:55.775309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:55.775309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:58:55.775309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:58:55.775309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:58:55.775309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:58:55.776285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:55.776285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:55.776285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:55.776285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:55.776285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:55.776285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:55.776285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:55.776285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:55.776285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:55.776285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:55.776285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:55.776285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:55.776285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:55.776285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:55.776285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:55.776285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:55.776285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:58:55.777127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:58:55.777127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:58:55.777127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:55.779351 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 227/462] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:58:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:55.790510 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f8a56-1c26-4abd-bc4e-ad561376bee8 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:55.863551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:55.908380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f8a56-1c26-4abd-bc4e-ad561376bee8 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:55.945525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e1259-fb8d-4c8f-adc0-182866cae4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:55.946571 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e1259-fb8d-4c8f-adc0-182866cae4c6 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:55.947066 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e1259-fb8d-4c8f-adc0-182866cae4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:55.947708 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e1259-fb8d-4c8f-adc0-182866cae4c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:55.948649 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e1259-fb8d-4c8f-adc0-182866cae4c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:55.949106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e1259-fb8d-4c8f-adc0-182866cae4c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:55.950772 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e1259-fb8d-4c8f-adc0-182866cae4c6 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:55.965285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 228/463] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:58:55.966590 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e1259-fb8d-4c8f-adc0-182866cae4c6 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:56.002267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7739966f-da2a-471c-86b4-3164186f79c6 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:56.073511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206ed34-1eac-4e9d-aaa2-ae8e869ab49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:56.074552 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206ed34-1eac-4e9d-aaa2-ae8e869ab49d None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:56.074552 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206ed34-1eac-4e9d-aaa2-ae8e869ab49d None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:56.074797 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206ed34-1eac-4e9d-aaa2-ae8e869ab49d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:56.086832 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a206ed34-1eac-4e9d-aaa2-ae8e869ab49d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:56.088295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a206ed34-1eac-4e9d-aaa2-ae8e869ab49d None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:56.098551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e1259-fb8d-4c8f-adc0-182866cae4c6 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:56.111165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-a206ed34-1eac-4e9d-aaa2-ae8e869ab49d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:58:56.111165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:56.111165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:56.111165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:56.111165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:56.111165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:56.111165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:56.111165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:56.111165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:56.111165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:56.111165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:56.111165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:56.111165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:58:56.111165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:58:56.111165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:58:56.111165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:58:56.112124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:56.112124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:56.112124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:56.112124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:56.112124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:56.112124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:56.112124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:56.112124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:56.112124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:56.112124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:56.112124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:56.112124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:56.112124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:56.112124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:56.112124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:56.112124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:56.112124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:58:56.112938 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:58:56.112938 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:58:56.112938 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:58:56.112938 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 236/464] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:58:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:58:56.113485 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206ed34-1eac-4e9d-aaa2-ae8e869ab49d None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:56.128234 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:56.143829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f11642-e4c3-49c6-857d-8307057efa35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:56.144471 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f11642-e4c3-49c6-857d-8307057efa35 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:56.144742 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f11642-e4c3-49c6-857d-8307057efa35 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:56.145211 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f11642-e4c3-49c6-857d-8307057efa35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:56.157159 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f11642-e4c3-49c6-857d-8307057efa35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:56.158975 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f11642-e4c3-49c6-857d-8307057efa35 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:56.167354 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:56.180182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-72f11642-e4c3-49c6-857d-8307057efa35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:58:56.180182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:56.180182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:56.180182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:56.180182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:56.180182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:56.180182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:56.180182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:56.180182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:56.180182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:56.180182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:56.180182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:56.180182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:58:56.180182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:58:56.180182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:58:56.180182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:58:56.181462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:56.181462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:56.181462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:56.181462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:56.181462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:56.181462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:56.181462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:56.181462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:56.181462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:56.181462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:56.181462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:56.181462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:56.181462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:56.181462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:56.181462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:56.181462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:56.181462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:58:56.182447 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:58:56.182447 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:58:56.182447 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:56.182447 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 229/465] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:58:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:56.183534 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d8911-f87b-4744-8f8a-8d880bef0fca None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:56.183980 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d8911-f87b-4744-8f8a-8d880bef0fca None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:56.184208 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d8911-f87b-4744-8f8a-8d880bef0fca None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:56.185859 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 230/466] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:56.186521 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d8911-f87b-4744-8f8a-8d880bef0fca None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:56.249395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206ed34-1eac-4e9d-aaa2-ae8e869ab49d None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:56.299102 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf59aaa-7ae3-4646-a082-0151bb35142f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:56.299922 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf59aaa-7ae3-4646-a082-0151bb35142f None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:56.300053 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf59aaa-7ae3-4646-a082-0151bb35142f None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:56.300243 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf59aaa-7ae3-4646-a082-0151bb35142f None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:56.300901 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf59aaa-7ae3-4646-a082-0151bb35142f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:56.302429 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf59aaa-7ae3-4646-a082-0151bb35142f None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:56.304317 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d8911-f87b-4744-8f8a-8d880bef0fca None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:56.334983 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 237/467] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 14:58:56 2022] POST /identity/v3/users => generated 248 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:58:56.334983 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf59aaa-7ae3-4646-a082-0151bb35142f None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:56.373976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9633051c-3368-4a48-95bf-0cfbce72d323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:56.374635 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633051c-3368-4a48-95bf-0cfbce72d323 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:56.375201 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633051c-3368-4a48-95bf-0cfbce72d323 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:56.375613 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633051c-3368-4a48-95bf-0cfbce72d323 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:56.376362 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9633051c-3368-4a48-95bf-0cfbce72d323 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:56.376670 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9633051c-3368-4a48-95bf-0cfbce72d323 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:56.381045 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9633051c-3368-4a48-95bf-0cfbce72d323 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:56.402486 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 231/468] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:58:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:56.409325 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39c150-cf93-4f4e-9386-40c7fadaec32 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:56.409626 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39c150-cf93-4f4e-9386-40c7fadaec32 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:56.409880 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39c150-cf93-4f4e-9386-40c7fadaec32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:56.431722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-aa39c150-cf93-4f4e-9386-40c7fadaec32 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:56.433075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-aa39c150-cf93-4f4e-9386-40c7fadaec32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:56.471022 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:56.488391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:56.491936 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf59aaa-7ae3-4646-a082-0151bb35142f None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:56.535512 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df584a-2f41-476e-b181-53a1cde5d573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:56.536281 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 232/469] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:56.537326 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39c150-cf93-4f4e-9386-40c7fadaec32 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:56.541329 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df584a-2f41-476e-b181-53a1cde5d573 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:56.541789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df584a-2f41-476e-b181-53a1cde5d573 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:56.542101 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df584a-2f41-476e-b181-53a1cde5d573 None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:56.542835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df584a-2f41-476e-b181-53a1cde5d573 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:56.543214 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df584a-2f41-476e-b181-53a1cde5d573 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:56.544688 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df584a-2f41-476e-b181-53a1cde5d573 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:56.565742 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 238/470] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:58:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:56.570938 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df584a-2f41-476e-b181-53a1cde5d573 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:56.747302 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39c150-cf93-4f4e-9386-40c7fadaec32 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:56.752408 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df584a-2f41-476e-b181-53a1cde5d573 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:56.796186 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e55090-9a29-48c5-8371-28d228f8332f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:56.796953 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e55090-9a29-48c5-8371-28d228f8332f None admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:56.797435 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e55090-9a29-48c5-8371-28d228f8332f None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:56.797723 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e55090-9a29-48c5-8371-28d228f8332f None admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:56.798573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e55090-9a29-48c5-8371-28d228f8332f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:56.800713 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e55090-9a29-48c5-8371-28d228f8332f None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:56.804171 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b3997-a889-43ec-ab69-82d837f9d154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:56.804795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b3997-a889-43ec-ab69-82d837f9d154 None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:56.805125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b3997-a889-43ec-ab69-82d837f9d154 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:56.805424 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b3997-a889-43ec-ab69-82d837f9d154 None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:56.806072 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497b3997-a889-43ec-ab69-82d837f9d154 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:56.807523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497b3997-a889-43ec-ab69-82d837f9d154 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:56.832751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 233/471] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 14:58:56 2022] POST /identity/v3/users => generated 246 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:58:56.834573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af34070-ebf2-477c-bf55-0aeb84995a73 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:56.835053 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af34070-ebf2-477c-bf55-0aeb84995a73 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:56.835529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af34070-ebf2-477c-bf55-0aeb84995a73 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:56.837441 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 234/472] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:56.841993 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 239/473] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 14:58:56 2022] POST /identity/v3/users => generated 249 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 14:58:56.847215 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab7c77-bd58-4803-8c80-ba21d1fba2e4 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:56.847840 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab7c77-bd58-4803-8c80-ba21d1fba2e4 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:56.848293 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab7c77-bd58-4803-8c80-ba21d1fba2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:56.858365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b3997-a889-43ec-ab69-82d837f9d154 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:56.884814 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-93ab7c77-bd58-4803-8c80-ba21d1fba2e4 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:56.886925 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-93ab7c77-bd58-4803-8c80-ba21d1fba2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:56.907475 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:56.923985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:57.049402 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 235/474] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:57.090213 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b3997-a889-43ec-ab69-82d837f9d154 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:57.130004 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422f3fb-3aa9-43d1-af9b-3b4154209cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:57.130656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422f3fb-3aa9-43d1-af9b-3b4154209cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:57.130930 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422f3fb-3aa9-43d1-af9b-3b4154209cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:57.131195 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422f3fb-3aa9-43d1-af9b-3b4154209cb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:57.145328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422f3fb-3aa9-43d1-af9b-3b4154209cb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:57.146802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422f3fb-3aa9-43d1-af9b-3b4154209cb8 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:57.161859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-e422f3fb-3aa9-43d1-af9b-3b4154209cb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:58:57.161859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:57.161859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:57.161859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:57.161859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:57.161859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:57.161859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:57.161859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:57.161859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:57.161859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:57.161859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:57.161859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:57.161859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:58:57.161859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:58:57.161859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:58:57.161859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:58:57.162792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:57.162792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:57.162792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:57.162792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:57.162792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:57.162792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:57.162792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:57.162792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:57.162792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:57.162792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:57.162792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:57.162792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:57.162792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:57.162792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:57.162792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:57.162792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:57.162792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:58:57.164202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:58:57.164202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:58:57.164202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:58:57.164202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 240/475] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:58:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:57.169110 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab7c77-bd58-4803-8c80-ba21d1fba2e4 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:57.287798 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab7c77-bd58-4803-8c80-ba21d1fba2e4 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:57.328484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e467bd-59f2-49a3-b0dc-24cfc440f465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:57.329014 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e467bd-59f2-49a3-b0dc-24cfc440f465 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:57.329313 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e467bd-59f2-49a3-b0dc-24cfc440f465 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:57.329577 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e467bd-59f2-49a3-b0dc-24cfc440f465 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:57.330223 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e467bd-59f2-49a3-b0dc-24cfc440f465 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:57.330677 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e467bd-59f2-49a3-b0dc-24cfc440f465 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:57.332055 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e467bd-59f2-49a3-b0dc-24cfc440f465 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:57.353477 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 236/476] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:57.354461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422f3fb-3aa9-43d1-af9b-3b4154209cb8 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:57.356599 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e467bd-59f2-49a3-b0dc-24cfc440f465 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:57.377594 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:57.437620 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:57.478306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e467bd-59f2-49a3-b0dc-24cfc440f465 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:57.485387 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422f3fb-3aa9-43d1-af9b-3b4154209cb8 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:57.514734 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6631b2-fa16-4e14-80e1-a5765262314d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:57.515347 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6631b2-fa16-4e14-80e1-a5765262314d None admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:57.515619 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6631b2-fa16-4e14-80e1-a5765262314d None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:57.515884 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6631b2-fa16-4e14-80e1-a5765262314d None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:57.516562 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6631b2-fa16-4e14-80e1-a5765262314d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:57.518219 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6631b2-fa16-4e14-80e1-a5765262314d None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:57.533724 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83f9f5-92eb-4e41-801d-db3b0cf62e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:57.534369 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83f9f5-92eb-4e41-801d-db3b0cf62e16 None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:57.534641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83f9f5-92eb-4e41-801d-db3b0cf62e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:57.534905 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83f9f5-92eb-4e41-801d-db3b0cf62e16 None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:57.535705 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e83f9f5-92eb-4e41-801d-db3b0cf62e16 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:57.537224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e83f9f5-92eb-4e41-801d-db3b0cf62e16 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:57.538476 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.sql.core [None req-aa6631b2-fa16-4e14-80e1-a5765262314d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 11 14:58:57.538476 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 11 14:58:57.538476 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [parameters: {'id': '5cf0b1868b9a4114a90cd9f07de21b1c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 11 14:58:57.538476 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81683) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 11 14:58:57.568985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-aa6631b2-fa16-4e14-80e1-a5765262314d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 11 14:58:57.568985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:57.568985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 14:58:57.568985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 14:58:57.568985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 14:58:57.568985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 14:58:57.568985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 14:58:57.568985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 14:58:57.568985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 14:58:57.568985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application conn.query(q) Sep 11 14:58:57.568985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 14:58:57.568985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 14:58:57.568985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 14:58:57.568985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application result.read() Sep 11 14:58:57.568985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 14:58:57.569987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 14:58:57.569987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 14:58:57.569987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 14:58:57.569987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 14:58:57.569987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 14:58:57.569987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 14:58:57.569987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 14:58:57.569987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 11 14:58:57.569987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:57.569987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 11 14:58:57.569987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:57.569987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:57.569987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 11 14:58:57.569987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 11 14:58:57.569987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 11 14:58:57.569987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return ref.to_dict() Sep 11 14:58:57.569987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 14:58:57.569987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application next(self.gen) Sep 11 14:58:57.575654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 11 14:58:57.575654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application yield resource Sep 11 14:58:57.575654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 14:58:57.575654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application next(self.gen) Sep 11 14:58:57.575654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 11 14:58:57.575654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self.session.rollback() Sep 11 14:58:57.575654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 11 14:58:57.575654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self.force_reraise() Sep 11 14:58:57.575654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 11 14:58:57.575654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise self.value Sep 11 14:58:57.575654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 11 14:58:57.575654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 11 14:58:57.575654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 11 14:58:57.575654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application session.commit() Sep 11 14:58:57.575654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 11 14:58:57.575654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 11 14:58:57.575654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 11 14:58:57.576691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self._prepare_impl() Sep 11 14:58:57.576691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 11 14:58:57.576691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self.session.flush() Sep 11 14:58:57.576691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 11 14:58:57.576691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self._flush(objects) Sep 11 14:58:57.576691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 11 14:58:57.576691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 11 14:58:57.576691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 11 14:58:57.576691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application compat.raise_( Sep 11 14:58:57.576691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 14:58:57.576691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception Sep 11 14:58:57.576691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 11 14:58:57.576691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application flush_context.execute() Sep 11 14:58:57.576691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 11 14:58:57.576691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rec.execute(self) Sep 11 14:58:57.576691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 11 14:58:57.576691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 11 14:58:57.577771 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 11 14:58:57.577771 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 11 14:58:57.577771 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 11 14:58:57.577771 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 11 14:58:57.577771 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 11 14:58:57.577771 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 11 14:58:57.577771 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 11 14:58:57.577771 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 11 14:58:57.577771 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 11 14:58:57.577771 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 11 14:58:57.577771 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 11 14:58:57.577771 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 11 14:58:57.577771 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 11 14:58:57.577771 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 11 14:58:57.577771 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 14:58:57.577771 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception Sep 11 14:58:57.578827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 14:58:57.578827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 14:58:57.578827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 14:58:57.578827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 14:58:57.578827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 14:58:57.578827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 14:58:57.578827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 14:58:57.578827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application conn.query(q) Sep 11 14:58:57.578827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 14:58:57.578827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 14:58:57.578827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 14:58:57.578827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application result.read() Sep 11 14:58:57.578827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 14:58:57.578827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 14:58:57.578827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 14:58:57.578827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 14:58:57.580455 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 14:58:57.580455 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 14:58:57.580455 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 14:58:57.580455 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 14:58:57.580455 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 11 14:58:57.580455 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 11 14:58:57.580455 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [parameters: {'id': '5cf0b1868b9a4114a90cd9f07de21b1c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 11 14:58:57.580455 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 11 14:58:57.580455 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:57.580455 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:58:57.580455 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:57.580455 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:57.580455 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:57.580455 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:57.580455 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:57.580455 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:57.582099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:57.582099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:57.582099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:57.582099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:57.582099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:57.582099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:57.582099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 11 14:58:57.582099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 11 14:58:57.582099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:57.582099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:57.582099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 11 14:58:57.582099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 11 14:58:57.582099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 11 14:58:57.582099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 11 14:58:57.582099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 11 14:58:57.582099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:57.583765 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 237/477] 10.222.0.44 () {60 vars in 1131 bytes} [Sun Sep 11 14:58:57 2022] POST /identity/v3/roles => generated 144 bytes in 214 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:58:57.585620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6631b2-fa16-4e14-80e1-a5765262314d None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:57.601507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 241/478] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 14:58:57 2022] POST /identity/v3/users => generated 248 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 14:58:57.718298 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:57.769176 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6631b2-fa16-4e14-80e1-a5765262314d None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:57.816082 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6257d6-03f0-4c88-9b97-9413990fcc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:57.816733 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6257d6-03f0-4c88-9b97-9413990fcc1c None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:57.817003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6257d6-03f0-4c88-9b97-9413990fcc1c None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:57.817308 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6257d6-03f0-4c88-9b97-9413990fcc1c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:57.832351 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6257d6-03f0-4c88-9b97-9413990fcc1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:57.837542 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-aa6257d6-03f0-4c88-9b97-9413990fcc1c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:58:57.837542 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:57.837542 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:57.837542 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:57.837542 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:57.837542 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:57.837542 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:57.837542 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:57.837542 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:57.837542 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:57.837542 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:57.837542 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:57.837542 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:57.837542 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:57.837542 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:57.837542 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:57.838380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:57.838380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:57.838380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:57.838380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:57.838380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:57.838380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:57.838380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:57.838380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:57.838380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:57.838380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:57.838380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:57.838380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:57.838380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:57.838380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:57.838380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:57.838380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:57.839596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:57.839596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:57.839596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:57.839596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:57.839596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:57.839596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:57.839596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:57.839596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:57.839596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:57.839596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:57.839596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 14:58:57.839596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:57.839596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 238/479] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 14:58:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:57.846447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83f9f5-92eb-4e41-801d-db3b0cf62e16 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:57.957274 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83f9f5-92eb-4e41-801d-db3b0cf62e16 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:57.982477 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:57.995987 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22740ec-755f-4572-995f-f143cbbef460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:57.996986 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22740ec-755f-4572-995f-f143cbbef460 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:57.997475 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22740ec-755f-4572-995f-f143cbbef460 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:57.997926 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22740ec-755f-4572-995f-f143cbbef460 None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:57.998881 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22740ec-755f-4572-995f-f143cbbef460 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:58:57.999366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22740ec-755f-4572-995f-f143cbbef460 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:58:58.001034 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22740ec-755f-4572-995f-f143cbbef460 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:58:58.017328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 242/480] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:58:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:58:59.245636 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2e8f5-2f19-4448-8874-235a579c1656 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:59.246275 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2e8f5-2f19-4448-8874-235a579c1656 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:59.246353 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2e8f5-2f19-4448-8874-235a579c1656 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:59.248116 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 239/481] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:59.254124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c87b9-5c41-48d1-89d9-db3def3b53de None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:59.255082 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c87b9-5c41-48d1-89d9-db3def3b53de None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:59.255611 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c87b9-5c41-48d1-89d9-db3def3b53de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:59.282202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-537c87b9-5c41-48d1-89d9-db3def3b53de None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:59.283425 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-537c87b9-5c41-48d1-89d9-db3def3b53de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:59.368000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 243/482] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:59.523977 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afb25d-f389-443b-8115-ffc2c1fce21c None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:59.524301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afb25d-f389-443b-8115-ffc2c1fce21c None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:59.524515 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afb25d-f389-443b-8115-ffc2c1fce21c None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:59.526171 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 240/483] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:59.531459 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15faf62-73b7-478b-81c1-f52289b4d13d None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:59.531746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15faf62-73b7-478b-81c1-f52289b4d13d None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:59.532015 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15faf62-73b7-478b-81c1-f52289b4d13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:59.553478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-d15faf62-73b7-478b-81c1-f52289b4d13d None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:59.554684 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-d15faf62-73b7-478b-81c1-f52289b4d13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:59.647532 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 244/484] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:58:59.737451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75afb25d-f389-443b-8115-ffc2c1fce21c None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:58:59.789257 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6c05e-b42d-416e-ba39-4faaf8703ae1 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:59.789257 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6c05e-b42d-416e-ba39-4faaf8703ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:59.789257 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6c05e-b42d-416e-ba39-4faaf8703ae1 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:59.790323 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 245/485] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:58:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:58:59.796113 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ed548-4f87-425a-bdc2-de7d928828cb None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:59.796682 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ed548-4f87-425a-bdc2-de7d928828cb None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:59.797196 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ed548-4f87-425a-bdc2-de7d928828cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:59.827552 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-af6ed548-4f87-425a-bdc2-de7d928828cb None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:58:59.829225 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-af6ed548-4f87-425a-bdc2-de7d928828cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:58:59.850565 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75afb25d-f389-443b-8115-ffc2c1fce21c None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:58:59.883705 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:59.893565 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:58:59.895909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214518b3-4c37-4557-bb23-d5830f95e0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:58:59.897644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214518b3-4c37-4557-bb23-d5830f95e0b8 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:58:59.898263 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214518b3-4c37-4557-bb23-d5830f95e0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:58:59.900096 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214518b3-4c37-4557-bb23-d5830f95e0b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:58:59.924945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214518b3-4c37-4557-bb23-d5830f95e0b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:58:59.927047 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 246/486] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:58:59.933197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-214518b3-4c37-4557-bb23-d5830f95e0b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:58:59.933197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:58:59.933197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:58:59.933197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:58:59.933197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:58:59.933197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:58:59.933197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:58:59.933197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:58:59.933197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:58:59.933197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:58:59.933197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:58:59.933197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:58:59.933197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:58:59.933197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:58:59.933197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:58:59.933197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:58:59.934241 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:58:59.934241 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:58:59.934241 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:58:59.934241 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:58:59.934241 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:58:59.934241 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:58:59.934241 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:58:59.934241 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:58:59.934241 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:58:59.934241 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:58:59.934241 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:58:59.934241 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:58:59.934241 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:58:59.934241 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:58:59.934241 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:58:59.934241 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:58:59.935025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:58:59.935025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:58:59.935025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:58:59.935025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:58:59.935025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:58:59.935025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:58:59.935025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:58:59.935025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:58:59.935025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:58:59.935025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:58:59.935025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:58:59.935025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:58:59.936144 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 241/487] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 14:58:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:58:59.937370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ed548-4f87-425a-bdc2-de7d928828cb None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:00.046268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214518b3-4c37-4557-bb23-d5830f95e0b8 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:00.076606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ed548-4f87-425a-bdc2-de7d928828cb None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:00.137296 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be26645-b0a4-4126-9500-43b78d3376e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:00.138607 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be26645-b0a4-4126-9500-43b78d3376e8 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:00.139217 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be26645-b0a4-4126-9500-43b78d3376e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:00.140449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be26645-b0a4-4126-9500-43b78d3376e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:00.142758 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be26645-b0a4-4126-9500-43b78d3376e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:00.143295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be26645-b0a4-4126-9500-43b78d3376e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:00.145866 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be26645-b0a4-4126-9500-43b78d3376e8 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:00.156368 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 247/488] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 14:58:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:00.163906 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be26645-b0a4-4126-9500-43b78d3376e8 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:00.219478 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214518b3-4c37-4557-bb23-d5830f95e0b8 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:00.241354 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:00.310723 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da1aac-ed0f-4987-9b47-eeec86a73568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:00.312680 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da1aac-ed0f-4987-9b47-eeec86a73568 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:00.313110 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da1aac-ed0f-4987-9b47-eeec86a73568 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:00.313546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da1aac-ed0f-4987-9b47-eeec86a73568 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:00.328457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da1aac-ed0f-4987-9b47-eeec86a73568 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:59:00.333803 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be26645-b0a4-4126-9500-43b78d3376e8 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:00.336050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-07da1aac-ed0f-4987-9b47-eeec86a73568 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:00.336050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:00.336050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:00.336050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:00.336050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:00.336050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:00.336050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:00.336050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:00.336050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:00.336050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:00.336050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:00.336050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:00.336050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:59:00.336050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:59:00.336050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:59:00.336050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:59:00.336915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:59:00.336915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:59:00.336915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:59:00.336915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:59:00.336915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:00.336915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:00.336915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:00.336915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:00.336915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:00.336915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:00.336915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:00.336915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:00.336915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:00.336915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:00.336915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:00.336915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:00.337747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:00.337747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:00.337747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:00.337747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:00.337747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:59:00.337747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:59:00.337747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:59:00.337747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:59:00.337747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:59:00.337747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:59:00.337747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:00.337747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:00.339011 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 242/489] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 14:59:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:59:00.339564 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da1aac-ed0f-4987-9b47-eeec86a73568 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:00.391338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43bf544-0237-479c-b0e2-8e36caeecd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:00.392445 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bf544-0237-479c-b0e2-8e36caeecd82 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:00.392930 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bf544-0237-479c-b0e2-8e36caeecd82 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:00.394140 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bf544-0237-479c-b0e2-8e36caeecd82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:00.408233 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43bf544-0237-479c-b0e2-8e36caeecd82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:00.409498 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:00.411871 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43bf544-0237-479c-b0e2-8e36caeecd82 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:00.430863 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-f43bf544-0237-479c-b0e2-8e36caeecd82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:00.430863 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:00.430863 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:00.430863 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:00.430863 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:00.430863 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:00.430863 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:00.430863 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:00.430863 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:00.430863 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:00.430863 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:00.430863 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:00.430863 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:00.430863 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:00.430863 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:00.430863 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:00.431830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:00.431830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:00.431830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:00.431830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:00.431830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:00.431830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:00.431830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:00.431830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:00.431830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:00.431830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:00.431830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:00.431830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:00.431830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:00.431830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:00.431830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:00.431830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:00.431830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:00.433318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:00.433318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:00.433318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:00.433318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 248/490] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:00.433318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43bf544-0237-479c-b0e2-8e36caeecd82 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:00.483251 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da1aac-ed0f-4987-9b47-eeec86a73568 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:00.523112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5403ee2-95bb-4733-934b-3e59d008f65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:00.524104 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5403ee2-95bb-4733-934b-3e59d008f65c None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:00.524647 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5403ee2-95bb-4733-934b-3e59d008f65c None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:00.525156 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5403ee2-95bb-4733-934b-3e59d008f65c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:00.539692 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5403ee2-95bb-4733-934b-3e59d008f65c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:59:00.547620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-b5403ee2-95bb-4733-934b-3e59d008f65c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:00.547620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:00.547620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:00.547620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:00.547620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:00.547620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:00.547620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:00.547620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:00.547620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:00.547620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:00.547620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:00.547620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:00.547620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:59:00.547620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:59:00.547620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:59:00.547620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:59:00.548573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:59:00.548573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:59:00.548573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:59:00.548573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:59:00.548573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:00.548573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:00.548573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:00.548573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:00.548573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:00.548573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:00.548573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:00.548573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:00.548573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:00.548573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:00.548573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:00.548573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:00.549391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:00.549391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:00.549391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:00.549391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:00.549391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:59:00.549391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:59:00.549391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:59:00.549391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:59:00.549391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:59:00.549391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:59:00.549391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:00.549391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:00.556772 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:00.557735 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 243/491] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 14:59:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:00.559078 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5403ee2-95bb-4733-934b-3e59d008f65c None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:00.567006 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43bf544-0237-479c-b0e2-8e36caeecd82 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:00.608440 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473fb5da-b64b-41a6-b2a4-325315afac07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:00.609058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fb5da-b64b-41a6-b2a4-325315afac07 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:00.609361 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fb5da-b64b-41a6-b2a4-325315afac07 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:00.609639 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fb5da-b64b-41a6-b2a4-325315afac07 None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:00.610277 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473fb5da-b64b-41a6-b2a4-325315afac07 None admin] RBAC: Adding query filter params (name=service) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:00.610577 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473fb5da-b64b-41a6-b2a4-325315afac07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:00.611953 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473fb5da-b64b-41a6-b2a4-325315afac07 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:00.626214 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 249/492] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 14:59:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:00.626214 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473fb5da-b64b-41a6-b2a4-325315afac07 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:00.679863 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5403ee2-95bb-4733-934b-3e59d008f65c None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:00.701246 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:00.734996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbe38d-2e76-4888-826d-0ebdece1e5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:00.735892 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbe38d-2e76-4888-826d-0ebdece1e5d8 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:00.736236 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbe38d-2e76-4888-826d-0ebdece1e5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:00.736578 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbe38d-2e76-4888-826d-0ebdece1e5d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:00.737344 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbe38d-2e76-4888-826d-0ebdece1e5d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:00.737674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbe38d-2e76-4888-826d-0ebdece1e5d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:00.739137 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbe38d-2e76-4888-826d-0ebdece1e5d8 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:00.754325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473fb5da-b64b-41a6-b2a4-325315afac07 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:00.757804 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 244/493] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:00.757804 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0642218-4e0a-4966-8ddf-954f61fe2217 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:00.757804 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0642218-4e0a-4966-8ddf-954f61fe2217 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:00.757804 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0642218-4e0a-4966-8ddf-954f61fe2217 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:00.757804 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 245/494] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:00.757804 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0642218-4e0a-4966-8ddf-954f61fe2217 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:00.801270 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3782e9-e5dc-47f5-ace3-854412176fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:00.801898 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3782e9-e5dc-47f5-ace3-854412176fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:00.802145 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3782e9-e5dc-47f5-ace3-854412176fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:00.802971 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3782e9-e5dc-47f5-ace3-854412176fb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:00.803097 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3782e9-e5dc-47f5-ace3-854412176fb8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:00.803385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3782e9-e5dc-47f5-ace3-854412176fb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:00.804758 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3782e9-e5dc-47f5-ace3-854412176fb8 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:00.815928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 250/495] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 14:59:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:00.815928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3782e9-e5dc-47f5-ace3-854412176fb8 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:00.836615 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:00.920404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0642218-4e0a-4966-8ddf-954f61fe2217 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:00.946436 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3782e9-e5dc-47f5-ace3-854412176fb8 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:00.963145 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec5367e-fe1c-4ae7-9935-0a1fd34b7533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:00.963805 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5367e-fe1c-4ae7-9935-0a1fd34b7533 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:00.964082 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5367e-fe1c-4ae7-9935-0a1fd34b7533 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:00.964355 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5367e-fe1c-4ae7-9935-0a1fd34b7533 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:00.979385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec5367e-fe1c-4ae7-9935-0a1fd34b7533 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:00.980960 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec5367e-fe1c-4ae7-9935-0a1fd34b7533 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:00.993127 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328dfe0-2a31-43ec-a6ce-769ebc3cd053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:00.993524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328dfe0-2a31-43ec-a6ce-769ebc3cd053 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:00.993711 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328dfe0-2a31-43ec-a6ce-769ebc3cd053 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:00.993982 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328dfe0-2a31-43ec-a6ce-769ebc3cd053 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:01.000374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-cec5367e-fe1c-4ae7-9935-0a1fd34b7533 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:01.000374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:01.000374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:01.000374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:01.000374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:01.000374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:01.000374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:01.000374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:01.000374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:01.000374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:01.000374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:01.000374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:01.000374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:01.000374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:01.000374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:01.000374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:01.001363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:01.001363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:01.001363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:01.001363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:01.001363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:01.001363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:01.001363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:01.001363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:01.001363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:01.001363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:01.001363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:01.001363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:01.001363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:01.001363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:01.001363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:01.001363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:01.001363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:01.002211 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:01.002211 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:01.002211 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:01.007482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6328dfe0-2a31-43ec-a6ce-769ebc3cd053 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:01.007592 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 246/496] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:01.007666 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6328dfe0-2a31-43ec-a6ce-769ebc3cd053 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:01.007732 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d1296-dc90-4ac3-afdc-ae0c4ae790c6 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:01.007732 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d1296-dc90-4ac3-afdc-ae0c4ae790c6 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:01.007732 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d1296-dc90-4ac3-afdc-ae0c4ae790c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:01.022716 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-6328dfe0-2a31-43ec-a6ce-769ebc3cd053 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 11 14:59:01.022716 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:01.022716 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:01.022716 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:01.022716 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:01.022716 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:01.022716 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:01.022716 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:01.022716 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:01.022716 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:01.022716 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:01.022716 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:01.022716 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:59:01.022716 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:59:01.022716 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:59:01.022716 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:59:01.028762 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:01.028762 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:01.028762 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:59:01.028762 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:01.028762 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:59:01.028762 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:01.028762 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:01.028762 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:01.028762 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:01.028762 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:01.028762 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:01.028762 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:01.028762 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:01.028762 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:01.028762 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:01.028762 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:01.028762 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:01.028762 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:01.030074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:01.030074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:01.030074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:59:01.030074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:59:01.030074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:59:01.030074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:59:01.030074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:59:01.030074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:59:01.030074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 11 14:59:01.030074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:01.030074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 251/497] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:59:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:01.030074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328dfe0-2a31-43ec-a6ce-769ebc3cd053 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:01.030689 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-d87d1296-dc90-4ac3-afdc-ae0c4ae790c6 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:01.030689 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-d87d1296-dc90-4ac3-afdc-ae0c4ae790c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:01.053768 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:01.119852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 247/498] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:59:01.120639 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d1296-dc90-4ac3-afdc-ae0c4ae790c6 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:01.125762 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:01.182839 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328dfe0-2a31-43ec-a6ce-769ebc3cd053 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:01.247773 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ade4cb-d931-4aa7-910a-cbf8d3738e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:01.249994 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ade4cb-d931-4aa7-910a-cbf8d3738e19 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:01.250748 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ade4cb-d931-4aa7-910a-cbf8d3738e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:01.251453 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ade4cb-d931-4aa7-910a-cbf8d3738e19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:01.271299 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ade4cb-d931-4aa7-910a-cbf8d3738e19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:01.272129 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d1296-dc90-4ac3-afdc-ae0c4ae790c6 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:01.274887 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ade4cb-d931-4aa7-910a-cbf8d3738e19 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:01.302111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-e5ade4cb-d931-4aa7-910a-cbf8d3738e19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:01.302111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:01.302111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:01.302111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:01.302111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:01.302111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:01.302111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:01.302111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:01.302111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:01.302111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:01.302111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:01.302111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:01.302111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:01.302111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:01.302111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:01.302111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:01.303958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:01.303958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:01.303958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:01.303958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:01.303958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:01.303958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:01.303958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:01.303958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:01.303958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:01.303958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:01.303958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:01.303958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:01.303958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:01.303958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:01.303958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:01.303958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:01.303958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:01.306588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:01.306588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:01.306588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:01.306588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 252/499] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:01.306588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ade4cb-d931-4aa7-910a-cbf8d3738e19 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:01.325496 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2252dd-9151-4759-b145-0be49bee9327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:01.326568 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2252dd-9151-4759-b145-0be49bee9327 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:01.327165 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2252dd-9151-4759-b145-0be49bee9327 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:01.327795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2252dd-9151-4759-b145-0be49bee9327 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:01.328895 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2252dd-9151-4759-b145-0be49bee9327 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:01.329425 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2252dd-9151-4759-b145-0be49bee9327 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:01.332015 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2252dd-9151-4759-b145-0be49bee9327 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:01.353332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 248/500] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:01.354840 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4362d9-abf2-4701-a295-145fff422f22 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:01.355177 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4362d9-abf2-4701-a295-145fff422f22 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:01.355591 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4362d9-abf2-4701-a295-145fff422f22 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:01.357299 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 249/501] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:01.362138 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4362d9-abf2-4701-a295-145fff422f22 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:01.381617 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:01.477762 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4362d9-abf2-4701-a295-145fff422f22 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:01.482828 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ade4cb-d931-4aa7-910a-cbf8d3738e19 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:01.518149 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:01.529393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c79ad-fae2-47ba-9642-ad5773016920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:01.530116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c79ad-fae2-47ba-9642-ad5773016920 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:01.530473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c79ad-fae2-47ba-9642-ad5773016920 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:01.530840 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c79ad-fae2-47ba-9642-ad5773016920 None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:01.531815 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c79ad-fae2-47ba-9642-ad5773016920 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:01.532177 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c79ad-fae2-47ba-9642-ad5773016920 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:01.532400 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d040c5-6e9f-4f54-b326-a18da673d729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:01.533043 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d040c5-6e9f-4f54-b326-a18da673d729 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:01.533349 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d040c5-6e9f-4f54-b326-a18da673d729 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:01.533619 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d040c5-6e9f-4f54-b326-a18da673d729 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:01.534049 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c79ad-fae2-47ba-9642-ad5773016920 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:01.534287 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d040c5-6e9f-4f54-b326-a18da673d729 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:01.534592 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d040c5-6e9f-4f54-b326-a18da673d729 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:01.536025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d040c5-6e9f-4f54-b326-a18da673d729 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:01.550912 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 250/502] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:01.551788 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d040c5-6e9f-4f54-b326-a18da673d729 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:01.583049 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 253/503] 10.222.0.44 () {56 vars in 1137 bytes} [Sun Sep 11 14:59:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:01.584474 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0135e32-dd10-4feb-ac2b-50a2a68fb6e6 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:01.584887 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0135e32-dd10-4feb-ac2b-50a2a68fb6e6 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:01.585310 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0135e32-dd10-4feb-ac2b-50a2a68fb6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:01.606912 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-d0135e32-dd10-4feb-ac2b-50a2a68fb6e6 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:01.608289 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-d0135e32-dd10-4feb-ac2b-50a2a68fb6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:01.651839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:01.680712 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d040c5-6e9f-4f54-b326-a18da673d729 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:01.687453 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 254/504] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:59:01.688337 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0135e32-dd10-4feb-ac2b-50a2a68fb6e6 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:01.738927 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbcb277-a608-4110-a1b9-8411a9472a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:01.740018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcb277-a608-4110-a1b9-8411a9472a73 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:01.740449 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcb277-a608-4110-a1b9-8411a9472a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:01.740883 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbcb277-a608-4110-a1b9-8411a9472a73 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:01.753324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbcb277-a608-4110-a1b9-8411a9472a73 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:01.755859 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbcb277-a608-4110-a1b9-8411a9472a73 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:01.778062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-9bbcb277-a608-4110-a1b9-8411a9472a73 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 14:59:01.778062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:01.778062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:01.778062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:01.778062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:01.778062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:01.778062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:01.778062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:01.778062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:01.778062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:01.778062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:01.778062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:01.778062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:59:01.778062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:59:01.778062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:59:01.778062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:59:01.780038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:01.780038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:01.780038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:59:01.780038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:01.780038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:59:01.780038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:01.780038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:01.780038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:01.780038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:01.780038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:01.780038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:01.780038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:01.780038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:01.780038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:01.780038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:01.780038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:01.780038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:01.780038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:01.785669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:01.785669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:01.785669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:59:01.785669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:59:01.785669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:59:01.785669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:59:01.785669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:59:01.785669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:59:01.785669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 14:59:01.785669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:01.785669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 251/505] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 14:59:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:01.785669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbcb277-a608-4110-a1b9-8411a9472a73 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:01.865663 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0135e32-dd10-4feb-ac2b-50a2a68fb6e6 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:01.889023 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:01.922285 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e307c15-4532-4989-8a1f-20283f60629d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:01.923093 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e307c15-4532-4989-8a1f-20283f60629d None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:01.923524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e307c15-4532-4989-8a1f-20283f60629d None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:01.923948 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e307c15-4532-4989-8a1f-20283f60629d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:01.941116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e307c15-4532-4989-8a1f-20283f60629d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:59:01.947320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbcb277-a608-4110-a1b9-8411a9472a73 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:01.950001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-8e307c15-4532-4989-8a1f-20283f60629d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:01.950001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:01.950001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:01.950001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:01.950001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:01.950001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:01.950001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:01.950001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:01.950001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:01.950001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:01.950001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:01.950001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:01.950001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:59:01.950001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:59:01.950001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:59:01.950001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:59:01.951590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:59:01.951590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:59:01.951590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:59:01.951590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:59:01.951590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:01.951590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:01.951590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:01.951590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:01.951590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:01.951590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:01.951590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:01.951590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:01.951590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:01.951590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:01.951590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:01.951590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:01.953117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:01.953117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:01.953117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:01.953117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:01.953117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:59:01.953117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:59:01.953117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:59:01.953117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:59:01.953117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:59:01.953117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:59:01.953117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:01.953117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:01.953117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 255/506] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 14:59:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:01.953117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e307c15-4532-4989-8a1f-20283f60629d None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:01.994587 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d03b5-5d8b-4980-99cf-9f38cff855b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:01.995307 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d03b5-5d8b-4980-99cf-9f38cff855b1 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:01.995647 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d03b5-5d8b-4980-99cf-9f38cff855b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:01.996089 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d03b5-5d8b-4980-99cf-9f38cff855b1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:01.998743 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:02.007377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d03b5-5d8b-4980-99cf-9f38cff855b1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:02.009996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d03b5-5d8b-4980-99cf-9f38cff855b1 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:02.024795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-413d03b5-5d8b-4980-99cf-9f38cff855b1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:59:02.024795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:02.024795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:02.024795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:02.024795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:02.024795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:02.024795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:02.024795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:02.024795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:02.024795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:02.024795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:02.024795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:02.024795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:59:02.024795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:59:02.024795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:59:02.024795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:59:02.026827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:02.026827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:02.026827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:59:02.026827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:02.026827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:59:02.026827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:02.026827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:02.026827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:02.026827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:02.026827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:02.026827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:02.026827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:02.026827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:02.026827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:02.026827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:02.026827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:02.026827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:02.026827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:02.028545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:02.028545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:02.028545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:59:02.028545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:59:02.028545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:59:02.028545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:59:02.028545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:59:02.028545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:59:02.028545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:59:02.028545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:02.028545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 252/507] 10.222.0.44 () {56 vars in 1093 bytes} [Sun Sep 11 14:59:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:02.028545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d03b5-5d8b-4980-99cf-9f38cff855b1 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:02.120241 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e307c15-4532-4989-8a1f-20283f60629d None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:02.159547 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d03b5-5d8b-4980-99cf-9f38cff855b1 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:02.160371 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762a856-8928-48c4-832f-7944f9c85662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:02.161100 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762a856-8928-48c4-832f-7944f9c85662 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:02.161580 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762a856-8928-48c4-832f-7944f9c85662 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:02.161947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762a856-8928-48c4-832f-7944f9c85662 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:02.201855 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66699ea6-2e10-40f7-bbed-3c5ad2acb934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:02.202593 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66699ea6-2e10-40f7-bbed-3c5ad2acb934 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:02.202966 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66699ea6-2e10-40f7-bbed-3c5ad2acb934 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:02.203377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66699ea6-2e10-40f7-bbed-3c5ad2acb934 None admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:02.204555 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66699ea6-2e10-40f7-bbed-3c5ad2acb934 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:02.204868 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66699ea6-2e10-40f7-bbed-3c5ad2acb934 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:02.206400 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66699ea6-2e10-40f7-bbed-3c5ad2acb934 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:02.213414 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:02.244311 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762a856-8928-48c4-832f-7944f9c85662 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:02.246159 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762a856-8928-48c4-832f-7944f9c85662 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:02.257766 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 253/508] 10.222.0.44 () {56 vars in 1139 bytes} [Sun Sep 11 14:59:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:02.258908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66699ea6-2e10-40f7-bbed-3c5ad2acb934 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:02.331360 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:02.394676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 256/509] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:59:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:59:02.394676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762a856-8928-48c4-832f-7944f9c85662 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:02.398517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66699ea6-2e10-40f7-bbed-3c5ad2acb934 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:02.441648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd64d5e-a41e-43d6-a73d-821f4d0c39b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:02.442579 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd64d5e-a41e-43d6-a73d-821f4d0c39b3 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:02.442984 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd64d5e-a41e-43d6-a73d-821f4d0c39b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:02.443370 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd64d5e-a41e-43d6-a73d-821f4d0c39b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:02.444325 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd64d5e-a41e-43d6-a73d-821f4d0c39b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:02.444818 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd64d5e-a41e-43d6-a73d-821f4d0c39b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:02.446385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd64d5e-a41e-43d6-a73d-821f4d0c39b3 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:02.456208 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 254/510] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 14:59:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:02.457198 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd64d5e-a41e-43d6-a73d-821f4d0c39b3 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:02.484401 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:02.519828 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762a856-8928-48c4-832f-7944f9c85662 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:02.570858 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6c723-3a0d-45e9-9d00-5604d60d45e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:02.571795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6c723-3a0d-45e9-9d00-5604d60d45e1 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:02.572204 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6c723-3a0d-45e9-9d00-5604d60d45e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:02.572659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6c723-3a0d-45e9-9d00-5604d60d45e1 None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:02.573762 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af6c723-3a0d-45e9-9d00-5604d60d45e1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:02.574316 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af6c723-3a0d-45e9-9d00-5604d60d45e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:02.576747 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af6c723-3a0d-45e9-9d00-5604d60d45e1 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:02.598306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd64d5e-a41e-43d6-a73d-821f4d0c39b3 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:02.602041 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 257/511] 10.222.0.44 () {56 vars in 1133 bytes} [Sun Sep 11 14:59:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:02.603045 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6c723-3a0d-45e9-9d00-5604d60d45e1 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:02.619098 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:02.644437 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36db07-e430-46b5-a807-449bae54156a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:02.645299 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36db07-e430-46b5-a807-449bae54156a None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:02.645662 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36db07-e430-46b5-a807-449bae54156a None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:02.646090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36db07-e430-46b5-a807-449bae54156a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:02.658148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36db07-e430-46b5-a807-449bae54156a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:02.659768 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36db07-e430-46b5-a807-449bae54156a None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:02.675474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-bb36db07-e430-46b5-a807-449bae54156a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:02.675474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:02.675474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:02.675474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:02.675474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:02.675474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:02.675474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:02.675474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:02.675474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:02.675474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:02.675474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:02.675474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:02.675474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:02.675474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:02.675474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:02.675474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:02.677312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:02.677312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:02.677312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:02.677312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:02.677312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:02.677312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:02.677312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:02.677312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:02.677312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:02.677312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:02.677312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:02.677312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:02.677312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:02.677312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:02.677312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:02.677312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:02.677312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:02.679546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:02.679546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:02.679546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:02.679546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 255/512] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:02.679546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36db07-e430-46b5-a807-449bae54156a None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:02.730854 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6c723-3a0d-45e9-9d00-5604d60d45e1 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:02.776390 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:02.784854 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1c65b-44f9-4ce5-871a-a0b3103ea688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:02.786225 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1c65b-44f9-4ce5-871a-a0b3103ea688 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:02.786665 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1c65b-44f9-4ce5-871a-a0b3103ea688 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:02.787104 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1c65b-44f9-4ce5-871a-a0b3103ea688 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:02.816344 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36db07-e430-46b5-a807-449bae54156a None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:02.865059 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a4495-f74b-4a87-abf1-3e2d0a778ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:02.865767 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a4495-f74b-4a87-abf1-3e2d0a778ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:02.866102 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a4495-f74b-4a87-abf1-3e2d0a778ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:02.866397 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a4495-f74b-4a87-abf1-3e2d0a778ca7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:02.879622 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a4495-f74b-4a87-abf1-3e2d0a778ca7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:02.881002 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1c65b-44f9-4ce5-871a-a0b3103ea688 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:02.881562 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a4495-f74b-4a87-abf1-3e2d0a778ca7 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:02.882739 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1c65b-44f9-4ce5-871a-a0b3103ea688 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:02.897749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-876a4495-f74b-4a87-abf1-3e2d0a778ca7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:02.897749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:02.897749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:02.897749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:02.897749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:02.897749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:02.897749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:02.897749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:02.897749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:02.897749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:02.897749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:02.897749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:02.897749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:02.897749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:02.897749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:02.897749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:02.899499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:02.899499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:02.899499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:02.899499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:02.899499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:02.899499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:02.899499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:02.899499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:02.899499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:02.899499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:02.899499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:02.899499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:02.899499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:02.899499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:02.899499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:02.899499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:02.899499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:02.901158 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:02.901158 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:02.901158 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:02.901158 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 256/513] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:02.901569 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a4495-f74b-4a87-abf1-3e2d0a778ca7 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:02.924316 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:03.005078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 258/514] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:59:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:59:03.006424 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1c65b-44f9-4ce5-871a-a0b3103ea688 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:03.030439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a4495-f74b-4a87-abf1-3e2d0a778ca7 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:03.044737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:03.071661 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c928834-73b4-4dcc-b4c5-35f4e563f3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:03.072604 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c928834-73b4-4dcc-b4c5-35f4e563f3db None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:03.073050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c928834-73b4-4dcc-b4c5-35f4e563f3db None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:03.073525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c928834-73b4-4dcc-b4c5-35f4e563f3db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:03.089891 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c928834-73b4-4dcc-b4c5-35f4e563f3db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:03.092478 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c928834-73b4-4dcc-b4c5-35f4e563f3db None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:03.116553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-0c928834-73b4-4dcc-b4c5-35f4e563f3db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:03.116553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:03.116553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:03.116553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:03.116553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:03.116553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:03.116553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:03.116553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:03.116553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:03.116553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:03.116553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:03.116553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:03.116553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:03.116553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:03.116553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:03.116553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:03.118301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:03.118301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:03.118301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:03.118301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:03.118301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:03.118301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:03.118301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:03.118301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:03.118301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:03.118301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:03.118301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:03.118301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:03.118301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:03.118301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:03.118301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:03.118301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:03.118301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:03.119920 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:03.119920 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:03.119920 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:03.119920 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 257/515] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:03.119920 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c928834-73b4-4dcc-b4c5-35f4e563f3db None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:03.151010 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1c65b-44f9-4ce5-871a-a0b3103ea688 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:03.202842 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073907b-b421-4468-9459-75ba2d420ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:03.203624 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073907b-b421-4468-9459-75ba2d420ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:03.203999 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073907b-b421-4468-9459-75ba2d420ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:03.204379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073907b-b421-4468-9459-75ba2d420ba5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:03.217367 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:03.260454 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c928834-73b4-4dcc-b4c5-35f4e563f3db None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:03.284271 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073907b-b421-4468-9459-75ba2d420ba5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:03.285988 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073907b-b421-4468-9459-75ba2d420ba5 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:03.302348 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029c4fa-558a-4fd9-a062-8ded2dfd2304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:03.303209 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029c4fa-558a-4fd9-a062-8ded2dfd2304 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:03.303573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029c4fa-558a-4fd9-a062-8ded2dfd2304 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:03.303936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029c4fa-558a-4fd9-a062-8ded2dfd2304 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:03.304888 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c029c4fa-558a-4fd9-a062-8ded2dfd2304 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:03.305330 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c029c4fa-558a-4fd9-a062-8ded2dfd2304 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:03.307008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c029c4fa-558a-4fd9-a062-8ded2dfd2304 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:03.320747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 258/516] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:03.321891 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029c4fa-558a-4fd9-a062-8ded2dfd2304 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:03.328945 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:03.416421 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 259/517] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:59:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:59:03.416421 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073907b-b421-4468-9459-75ba2d420ba5 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:03.455075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029c4fa-558a-4fd9-a062-8ded2dfd2304 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:03.490650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:03.500834 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f2f23-64b0-4b3f-b290-8f31fb4b7cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:03.501815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f2f23-64b0-4b3f-b290-8f31fb4b7cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:03.502229 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f2f23-64b0-4b3f-b290-8f31fb4b7cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:03.502854 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f2f23-64b0-4b3f-b290-8f31fb4b7cb3 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:03.503874 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8f2f23-64b0-4b3f-b290-8f31fb4b7cb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:03.504274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8f2f23-64b0-4b3f-b290-8f31fb4b7cb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:03.506547 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8f2f23-64b0-4b3f-b290-8f31fb4b7cb3 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:03.529538 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 259/518] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:03.529538 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f2f23-64b0-4b3f-b290-8f31fb4b7cb3 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:03.551246 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073907b-b421-4468-9459-75ba2d420ba5 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:03.585485 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:03.592079 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109db05-bc5d-4c87-9a9c-0d62615cb1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:03.592765 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109db05-bc5d-4c87-9a9c-0d62615cb1a9 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:03.593159 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109db05-bc5d-4c87-9a9c-0d62615cb1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:03.593515 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109db05-bc5d-4c87-9a9c-0d62615cb1a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:03.605580 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6109db05-bc5d-4c87-9a9c-0d62615cb1a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:03.607247 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6109db05-bc5d-4c87-9a9c-0d62615cb1a9 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:03.624033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-6109db05-bc5d-4c87-9a9c-0d62615cb1a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:03.624033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:03.624033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:03.624033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:03.624033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:03.624033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:03.624033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:03.624033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:03.624033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:03.624033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:03.624033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:03.624033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:03.624033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:03.624033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:03.624033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:03.624033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:03.626026 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:03.626026 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:03.626026 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:03.626026 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:03.626026 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:03.626026 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:03.626026 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:03.626026 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:03.626026 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:03.626026 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:03.626026 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:03.626026 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:03.626026 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:03.626026 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:03.626026 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:03.626026 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:03.626026 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:03.627675 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:03.627675 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:03.627675 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:03.627675 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 260/519] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:03.627675 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109db05-bc5d-4c87-9a9c-0d62615cb1a9 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:03.654322 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f2f23-64b0-4b3f-b290-8f31fb4b7cb3 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:03.699730 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad5e10-dc1c-4b98-97f2-ebb37510721f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:03.700671 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad5e10-dc1c-4b98-97f2-ebb37510721f None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:03.701118 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad5e10-dc1c-4b98-97f2-ebb37510721f None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:03.701574 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad5e10-dc1c-4b98-97f2-ebb37510721f None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:03.702632 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad5e10-dc1c-4b98-97f2-ebb37510721f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:03.703087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad5e10-dc1c-4b98-97f2-ebb37510721f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:03.705488 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad5e10-dc1c-4b98-97f2-ebb37510721f None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:03.724749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 260/520] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:03.725969 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad5e10-dc1c-4b98-97f2-ebb37510721f None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:03.769058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109db05-bc5d-4c87-9a9c-0d62615cb1a9 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:03.808579 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:03.821071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b603900-b66e-4bf0-b1ef-f35d1b978dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:03.821824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b603900-b66e-4bf0-b1ef-f35d1b978dd3 None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:03.822239 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b603900-b66e-4bf0-b1ef-f35d1b978dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:03.822556 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b603900-b66e-4bf0-b1ef-f35d1b978dd3 None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:03.823248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b603900-b66e-4bf0-b1ef-f35d1b978dd3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:03.824913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b603900-b66e-4bf0-b1ef-f35d1b978dd3 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:03.852951 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 261/521] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 14:59:03 2022] POST /identity/v3/users => generated 247 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:59:03.854753 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b603900-b66e-4bf0-b1ef-f35d1b978dd3 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:03.891710 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad5e10-dc1c-4b98-97f2-ebb37510721f None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:03.936202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:03.937938 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d264dc76-3bc9-4de6-a73c-356111deecf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:03.938997 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d264dc76-3bc9-4de6-a73c-356111deecf7 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:03.939667 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d264dc76-3bc9-4de6-a73c-356111deecf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:03.940006 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d264dc76-3bc9-4de6-a73c-356111deecf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:03.954228 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d264dc76-3bc9-4de6-a73c-356111deecf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:03.955778 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d264dc76-3bc9-4de6-a73c-356111deecf7 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:03.974820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-d264dc76-3bc9-4de6-a73c-356111deecf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:03.974820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:03.974820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:03.974820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:03.974820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:03.974820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:03.974820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:03.974820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:03.974820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:03.974820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:03.974820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:03.974820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:03.974820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:03.974820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:03.974820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:03.974820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:03.976377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:03.976377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:03.976377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:03.976377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:03.976377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:03.976377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:03.976377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:03.976377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:03.976377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:03.976377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:03.976377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:03.976377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:03.976377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:03.976377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:03.976377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:03.976377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:03.976377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:03.977848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:03.977848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:03.977848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:03.981275 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 261/522] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:03.982288 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d264dc76-3bc9-4de6-a73c-356111deecf7 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:04.058330 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b603900-b66e-4bf0-b1ef-f35d1b978dd3 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:04.116631 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f83460-9e50-4155-a43c-aff28690b687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:04.117518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f83460-9e50-4155-a43c-aff28690b687 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:04.117924 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f83460-9e50-4155-a43c-aff28690b687 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:04.120100 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f83460-9e50-4155-a43c-aff28690b687 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:04.130752 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d264dc76-3bc9-4de6-a73c-356111deecf7 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:04.132621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:04.134868 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f83460-9e50-4155-a43c-aff28690b687 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:04.136319 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f83460-9e50-4155-a43c-aff28690b687 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:04.154314 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-e9f83460-9e50-4155-a43c-aff28690b687 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:04.154314 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:04.154314 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:04.154314 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:04.154314 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:04.154314 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:04.154314 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:04.154314 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:04.154314 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:04.154314 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:04.154314 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:04.154314 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:04.154314 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:59:04.154314 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:59:04.154314 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:59:04.155958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:59:04.155958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:04.155958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:04.155958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:04.155958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:04.155958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:04.155958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:04.155958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:04.155958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:04.155958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:04.155958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:04.155958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:04.155958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:04.155958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:04.155958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:04.155958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:04.155958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:04.159050 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:59:04.159050 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:59:04.159050 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:59:04.159050 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:59:04.159050 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:59:04.159050 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:59:04.159050 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:04.159050 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:04.159050 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 262/523] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:59:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:04.159050 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f83460-9e50-4155-a43c-aff28690b687 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:04.178607 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d4ce1-c99b-48c2-a6b2-0585a9c25b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:04.179663 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d4ce1-c99b-48c2-a6b2-0585a9c25b6c None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:04.179950 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d4ce1-c99b-48c2-a6b2-0585a9c25b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:04.180352 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d4ce1-c99b-48c2-a6b2-0585a9c25b6c None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:04.181418 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d4ce1-c99b-48c2-a6b2-0585a9c25b6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:04.181851 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d4ce1-c99b-48c2-a6b2-0585a9c25b6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:04.183748 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d4ce1-c99b-48c2-a6b2-0585a9c25b6c None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:04.199822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 262/524] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:04.203084 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d4ce1-c99b-48c2-a6b2-0585a9c25b6c None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:04.285845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:04.290057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f83460-9e50-4155-a43c-aff28690b687 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:04.327345 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a57c7b-129d-418b-a5f5-eb71f905c011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:04.327987 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a57c7b-129d-418b-a5f5-eb71f905c011 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:04.328264 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a57c7b-129d-418b-a5f5-eb71f905c011 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:04.328530 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a57c7b-129d-418b-a5f5-eb71f905c011 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:04.336744 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a57c7b-129d-418b-a5f5-eb71f905c011 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:04.338291 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a57c7b-129d-418b-a5f5-eb71f905c011 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:04.350850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-d0a57c7b-129d-418b-a5f5-eb71f905c011 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 14:59:04.350850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:04.350850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:04.350850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:04.350850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:04.350850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:04.350850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:04.350850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:04.350850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:04.350850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:04.350850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:04.350850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:04.350850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:59:04.350850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:59:04.350850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:59:04.350850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:59:04.352853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:04.352853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:04.352853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:59:04.352853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:04.352853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:59:04.352853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:04.352853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:04.352853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:04.352853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:04.352853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:04.352853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:04.352853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:04.352853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:04.352853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:04.352853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:04.352853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:04.352853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:04.352853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:04.354914 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:04.354914 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:04.354914 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:59:04.354914 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:59:04.354914 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:59:04.354914 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:59:04.354914 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:59:04.354914 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:59:04.354914 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 14:59:04.354914 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:04.356851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 263/525] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:59:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:04.357801 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a57c7b-129d-418b-a5f5-eb71f905c011 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:04.367310 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d4ce1-c99b-48c2-a6b2-0585a9c25b6c None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:04.432725 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040a90f-9f1e-4833-9df3-7189a6628406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:04.434422 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040a90f-9f1e-4833-9df3-7189a6628406 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:04.435145 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040a90f-9f1e-4833-9df3-7189a6628406 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:04.435805 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040a90f-9f1e-4833-9df3-7189a6628406 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:04.436994 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d040a90f-9f1e-4833-9df3-7189a6628406 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:04.437639 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d040a90f-9f1e-4833-9df3-7189a6628406 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:04.440763 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d040a90f-9f1e-4833-9df3-7189a6628406 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:04.455701 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 263/526] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:04.457301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040a90f-9f1e-4833-9df3-7189a6628406 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:04.481199 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a57c7b-129d-418b-a5f5-eb71f905c011 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:04.486462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:04.537357 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b78900-9c3d-48a5-a012-ca4014688650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:04.538351 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b78900-9c3d-48a5-a012-ca4014688650 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:04.538835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b78900-9c3d-48a5-a012-ca4014688650 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:04.539319 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b78900-9c3d-48a5-a012-ca4014688650 None admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:04.540596 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b78900-9c3d-48a5-a012-ca4014688650 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:04.541159 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b78900-9c3d-48a5-a012-ca4014688650 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:04.543988 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b78900-9c3d-48a5-a012-ca4014688650 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:04.570414 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040a90f-9f1e-4833-9df3-7189a6628406 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:04.576705 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 264/527] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 14:59:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:04.578329 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b78900-9c3d-48a5-a012-ca4014688650 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:04.617573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd4707-2ea9-4b5d-abfe-a95d9ea17214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:04.618148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd4707-2ea9-4b5d-abfe-a95d9ea17214 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:04.618438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd4707-2ea9-4b5d-abfe-a95d9ea17214 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:04.618705 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd4707-2ea9-4b5d-abfe-a95d9ea17214 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:04.619597 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:04.626555 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd4707-2ea9-4b5d-abfe-a95d9ea17214 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:04.628040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd4707-2ea9-4b5d-abfe-a95d9ea17214 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:04.642391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-dbdd4707-2ea9-4b5d-abfe-a95d9ea17214 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:04.642391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:04.642391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:04.642391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:04.642391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:04.642391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:04.642391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:04.642391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:04.642391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:04.642391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:04.642391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:04.642391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:04.642391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:59:04.642391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:59:04.642391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:59:04.643303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:59:04.643303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:04.643303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:04.643303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:04.643303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:04.643303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:04.643303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:04.643303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:04.643303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:04.643303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:04.643303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:04.643303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:04.643303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:04.643303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:04.643303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:04.643303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:04.643303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:04.644250 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:59:04.644250 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:59:04.644250 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:59:04.644250 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:59:04.644250 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:59:04.644250 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:59:04.644250 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:04.644250 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:04.646108 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 264/528] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:59:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:04.647174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd4707-2ea9-4b5d-abfe-a95d9ea17214 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:04.731526 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b78900-9c3d-48a5-a012-ca4014688650 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:04.756316 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:04.776963 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82977df-5f57-439e-81a9-2bbea0f211e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:04.777879 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82977df-5f57-439e-81a9-2bbea0f211e5 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:04.778284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82977df-5f57-439e-81a9-2bbea0f211e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:04.778553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd4707-2ea9-4b5d-abfe-a95d9ea17214 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:04.778741 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82977df-5f57-439e-81a9-2bbea0f211e5 None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:04.779607 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82977df-5f57-439e-81a9-2bbea0f211e5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:04.780108 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82977df-5f57-439e-81a9-2bbea0f211e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:04.782281 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82977df-5f57-439e-81a9-2bbea0f211e5 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:04.803556 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 265/529] 10.222.0.44 () {56 vars in 1137 bytes} [Sun Sep 11 14:59:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:04.804549 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82977df-5f57-439e-81a9-2bbea0f211e5 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:04.823453 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656ec5f-a12c-4ee8-80d3-129e9f19b5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:04.824269 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656ec5f-a12c-4ee8-80d3-129e9f19b5ee None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:04.824579 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656ec5f-a12c-4ee8-80d3-129e9f19b5ee None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:04.824820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656ec5f-a12c-4ee8-80d3-129e9f19b5ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:04.833122 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0656ec5f-a12c-4ee8-80d3-129e9f19b5ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:04.835016 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0656ec5f-a12c-4ee8-80d3-129e9f19b5ee None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:04.848484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-0656ec5f-a12c-4ee8-80d3-129e9f19b5ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:04.848484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:04.848484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:04.848484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:04.848484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:04.848484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:04.848484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:04.848484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:04.848484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:04.848484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:04.848484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:04.848484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:04.848484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:59:04.848484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:59:04.848484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:59:04.856365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:59:04.856365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:04.856365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:04.856365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:04.856365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:04.856365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:04.856365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:04.856365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:04.856365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:04.856365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:04.856365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:04.856365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:04.856365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:04.856365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:04.856365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:04.856365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:04.856365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:04.858438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:59:04.858438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:59:04.858438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:59:04.858438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:59:04.858438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:59:04.858438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:59:04.858438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:04.858438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:04.858438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 265/530] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:59:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:04.858438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656ec5f-a12c-4ee8-80d3-129e9f19b5ee None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:04.921310 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:04.955118 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82977df-5f57-439e-81a9-2bbea0f211e5 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:04.978251 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656ec5f-a12c-4ee8-80d3-129e9f19b5ee None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:04.996428 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c293c1e-050f-42fa-976d-924aa199dab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:04.997044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c293c1e-050f-42fa-976d-924aa199dab2 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:04.997356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c293c1e-050f-42fa-976d-924aa199dab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:04.997627 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c293c1e-050f-42fa-976d-924aa199dab2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:04.998378 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c293c1e-050f-42fa-976d-924aa199dab2 None admin] RBAC: Adding query filter params (role.id=ea3b2f1eb283418b8823cb4fff80edba, scope.project.id=70ac515938a24c45ae6302dbac46a1ab, user.id=a1fe5293447346eaafd16818e5bd12f4) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:04.998681 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c293c1e-050f-42fa-976d-924aa199dab2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:05.000154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c293c1e-050f-42fa-976d-924aa199dab2 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:05.006400 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 266/531] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:59:04 2022] GET /identity/v3/role_assignments?user.id=a1fe5293447346eaafd16818e5bd12f4&scope.project.id=70ac515938a24c45ae6302dbac46a1ab&role.id=ea3b2f1eb283418b8823cb4fff80edba => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:05.007259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c293c1e-050f-42fa-976d-924aa199dab2 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:05.027956 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a4ee6-9c12-456f-b34d-eb9a689538ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:05.029010 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a4ee6-9c12-456f-b34d-eb9a689538ee None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:05.029509 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a4ee6-9c12-456f-b34d-eb9a689538ee None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:05.029953 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a4ee6-9c12-456f-b34d-eb9a689538ee None admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:05.030998 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a4ee6-9c12-456f-b34d-eb9a689538ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:05.031312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a4ee6-9c12-456f-b34d-eb9a689538ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:05.032836 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a4ee6-9c12-456f-b34d-eb9a689538ee None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:05.060125 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 266/532] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 14:59:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:05.060125 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a4ee6-9c12-456f-b34d-eb9a689538ee None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:05.060947 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:05.181764 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c293c1e-050f-42fa-976d-924aa199dab2 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:05.199692 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a4ee6-9c12-456f-b34d-eb9a689538ee None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:05.241956 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb1388-6e99-44df-a4a1-853ef667759b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:05.242996 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb1388-6e99-44df-a4a1-853ef667759b None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:05.243469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb1388-6e99-44df-a4a1-853ef667759b None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:05.243956 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb1388-6e99-44df-a4a1-853ef667759b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:05.268175 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:05.273983 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe6d4b-ed96-40a0-92f1-74a5c4ae494c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:05.275986 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe6d4b-ed96-40a0-92f1-74a5c4ae494c None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:05.277578 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe6d4b-ed96-40a0-92f1-74a5c4ae494c None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:05.277861 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe6d4b-ed96-40a0-92f1-74a5c4ae494c None admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:05.280251 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe6d4b-ed96-40a0-92f1-74a5c4ae494c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:05.281369 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe6d4b-ed96-40a0-92f1-74a5c4ae494c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:05.282861 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe6d4b-ed96-40a0-92f1-74a5c4ae494c None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:05.300370 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 267/533] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 14:59:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:05.302110 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe6d4b-ed96-40a0-92f1-74a5c4ae494c None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:05.357929 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb1388-6e99-44df-a4a1-853ef667759b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:05.360152 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb1388-6e99-44df-a4a1-853ef667759b None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:05.400855 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:05.453998 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe6d4b-ed96-40a0-92f1-74a5c4ae494c None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:05.488041 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 267/534] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:59:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:59:05.489554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb1388-6e99-44df-a4a1-853ef667759b None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:05.495509 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6626aef-8f8c-484f-8a2f-b7bbd560f268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:05.496939 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6626aef-8f8c-484f-8a2f-b7bbd560f268 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:05.497479 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6626aef-8f8c-484f-8a2f-b7bbd560f268 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:05.497964 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6626aef-8f8c-484f-8a2f-b7bbd560f268 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:05.498928 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6626aef-8f8c-484f-8a2f-b7bbd560f268 None admin] RBAC: Adding query filter params (role.id=ea3b2f1eb283418b8823cb4fff80edba, scope.project.id=70ac515938a24c45ae6302dbac46a1ab, user.id=697b73f182e6419a84e0c8c6b0fc0c06) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:05.499436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6626aef-8f8c-484f-8a2f-b7bbd560f268 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:05.501247 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6626aef-8f8c-484f-8a2f-b7bbd560f268 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:05.513495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 268/535] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:59:05 2022] GET /identity/v3/role_assignments?user.id=697b73f182e6419a84e0c8c6b0fc0c06&scope.project.id=70ac515938a24c45ae6302dbac46a1ab&role.id=ea3b2f1eb283418b8823cb4fff80edba => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:05.513495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6626aef-8f8c-484f-8a2f-b7bbd560f268 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:05.587520 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:05.718089 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb1388-6e99-44df-a4a1-853ef667759b None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:05.743075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6626aef-8f8c-484f-8a2f-b7bbd560f268 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:05.802231 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97569f32-c501-4c87-9ec5-9faf4417ad50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:05.802979 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97569f32-c501-4c87-9ec5-9faf4417ad50 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:05.803248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97569f32-c501-4c87-9ec5-9faf4417ad50 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:05.803513 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97569f32-c501-4c87-9ec5-9faf4417ad50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:05.804351 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97569f32-c501-4c87-9ec5-9faf4417ad50 None admin] RBAC: Adding query filter params (role.id=ea3b2f1eb283418b8823cb4fff80edba, scope.project.id=70ac515938a24c45ae6302dbac46a1ab, user.id=453297dac64046d19fc1c331d2d85abc) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:05.804715 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97569f32-c501-4c87-9ec5-9faf4417ad50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:05.806517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97569f32-c501-4c87-9ec5-9faf4417ad50 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:05.809398 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:05.828873 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 268/536] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:59:05 2022] GET /identity/v3/role_assignments?user.id=453297dac64046d19fc1c331d2d85abc&scope.project.id=70ac515938a24c45ae6302dbac46a1ab&role.id=ea3b2f1eb283418b8823cb4fff80edba => generated 259 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:05.830903 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed48acb-c812-42bb-aa62-49ca3a2d31c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:05.831990 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed48acb-c812-42bb-aa62-49ca3a2d31c7 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:05.832439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed48acb-c812-42bb-aa62-49ca3a2d31c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:05.832890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed48acb-c812-42bb-aa62-49ca3a2d31c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:05.856700 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed48acb-c812-42bb-aa62-49ca3a2d31c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:05.859393 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed48acb-c812-42bb-aa62-49ca3a2d31c7 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:05.884958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-aed48acb-c812-42bb-aa62-49ca3a2d31c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:05.884958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:05.884958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:05.884958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:05.884958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:05.884958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:05.884958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:05.884958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:05.884958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:05.884958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:05.884958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:05.884958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:05.884958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:05.884958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:05.884958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:05.884958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:05.886380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:05.886380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:05.886380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:05.886380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:05.886380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:05.886380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:05.886380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:05.886380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:05.886380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:05.886380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:05.886380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:05.886380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:05.886380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:05.886380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:05.886380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:05.886380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:05.886380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:05.887389 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:05.887389 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:05.887389 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:05.887389 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 269/537] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:05.892612 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97569f32-c501-4c87-9ec5-9faf4417ad50 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:06.096434 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97569f32-c501-4c87-9ec5-9faf4417ad50 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:06.106715 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:06.142949 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4a0ef-c8a3-4a86-88d1-4e683c9c7f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:06.143584 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4a0ef-c8a3-4a86-88d1-4e683c9c7f06 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:06.143855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4a0ef-c8a3-4a86-88d1-4e683c9c7f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:06.144132 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4a0ef-c8a3-4a86-88d1-4e683c9c7f06 None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:06.144824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4a0ef-c8a3-4a86-88d1-4e683c9c7f06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:06.145180 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4a0ef-c8a3-4a86-88d1-4e683c9c7f06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:06.146670 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4a0ef-c8a3-4a86-88d1-4e683c9c7f06 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:06.161714 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 269/538] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:06.167559 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed48acb-c812-42bb-aa62-49ca3a2d31c7 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:06.312468 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed48acb-c812-42bb-aa62-49ca3a2d31c7 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:06.355218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ce7e9-2a0b-4c53-87f9-6cf68abe84c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:06.355869 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ce7e9-2a0b-4c53-87f9-6cf68abe84c9 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:06.356143 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ce7e9-2a0b-4c53-87f9-6cf68abe84c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:06.356411 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ce7e9-2a0b-4c53-87f9-6cf68abe84c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:06.364033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ce7e9-2a0b-4c53-87f9-6cf68abe84c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:06.365531 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ce7e9-2a0b-4c53-87f9-6cf68abe84c9 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:06.378061 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-ab2ce7e9-2a0b-4c53-87f9-6cf68abe84c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:06.378061 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:06.378061 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:06.378061 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:06.378061 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:06.378061 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:06.378061 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:06.378061 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:06.378061 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:06.378061 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:06.378061 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:06.378061 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:06.378061 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:59:06.378061 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:59:06.378061 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:59:06.379027 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:59:06.379027 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:06.379027 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:06.379027 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:06.379027 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:06.379027 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:06.379027 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:06.379027 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:06.379027 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:06.379027 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:06.379027 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:06.379027 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:06.379027 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:06.379027 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:06.379027 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:06.379027 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:06.379027 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:06.379860 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:59:06.379860 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:59:06.379860 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:59:06.379860 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:59:06.379860 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:59:06.379860 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:59:06.379860 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:06.379860 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:06.379860 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 270/539] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:59:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:06.386957 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4a0ef-c8a3-4a86-88d1-4e683c9c7f06 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:06.439967 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20263de2-355c-4ea4-b3e2-b8f9a7c7d16a None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:06.440329 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20263de2-355c-4ea4-b3e2-b8f9a7c7d16a None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:06.440501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20263de2-355c-4ea4-b3e2-b8f9a7c7d16a None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:06.442210 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 271/540] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:59:06.461863 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7f2c6-b02d-4aa2-8350-aea14cc8514b None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:06.462221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7f2c6-b02d-4aa2-8350-aea14cc8514b None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:06.462401 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7f2c6-b02d-4aa2-8350-aea14cc8514b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:06.482448 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-aaf7f2c6-b02d-4aa2-8350-aea14cc8514b None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:06.487259 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-aaf7f2c6-b02d-4aa2-8350-aea14cc8514b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:06.526196 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:06.566115 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4a0ef-c8a3-4a86-88d1-4e683c9c7f06 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:06.616028 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 272/541] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:59:06.641036 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c103fd7-303c-4667-96b4-fa557771942c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:06.642462 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c103fd7-303c-4667-96b4-fa557771942c None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:06.642941 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c103fd7-303c-4667-96b4-fa557771942c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:06.643441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c103fd7-303c-4667-96b4-fa557771942c None admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:06.644378 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c103fd7-303c-4667-96b4-fa557771942c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:06.644901 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c103fd7-303c-4667-96b4-fa557771942c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:06.646669 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c103fd7-303c-4667-96b4-fa557771942c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:06.665808 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 270/542] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 14:59:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:06.688265 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7f2c6-b02d-4aa2-8350-aea14cc8514b None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:06.839465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7f2c6-b02d-4aa2-8350-aea14cc8514b None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:06.895669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22b9e6-ae61-4b2a-95c3-f0d1cb7633d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:06.896689 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22b9e6-ae61-4b2a-95c3-f0d1cb7633d1 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:06.897218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22b9e6-ae61-4b2a-95c3-f0d1cb7633d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:06.897701 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22b9e6-ae61-4b2a-95c3-f0d1cb7633d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:06.898676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22b9e6-ae61-4b2a-95c3-f0d1cb7633d1 None admin] RBAC: Adding query filter params (role.id=ea3b2f1eb283418b8823cb4fff80edba, scope.project.id=70ac515938a24c45ae6302dbac46a1ab, user.id=8b97e2542db54900a99f6f44b78a5e24) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:06.899188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22b9e6-ae61-4b2a-95c3-f0d1cb7633d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:06.900885 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22b9e6-ae61-4b2a-95c3-f0d1cb7633d1 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:06.914342 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 273/543] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:59:06 2022] GET /identity/v3/role_assignments?user.id=8b97e2542db54900a99f6f44b78a5e24&scope.project.id=70ac515938a24c45ae6302dbac46a1ab&role.id=ea3b2f1eb283418b8823cb4fff80edba => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:59:07.008588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c103fd7-303c-4667-96b4-fa557771942c None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:07.048170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:07.201952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c103fd7-303c-4667-96b4-fa557771942c None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:07.266958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01957625-ccd3-4b10-b370-41ab73e282f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:07.267990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01957625-ccd3-4b10-b370-41ab73e282f5 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:07.268436 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01957625-ccd3-4b10-b370-41ab73e282f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:07.268883 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01957625-ccd3-4b10-b370-41ab73e282f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:07.295780 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01957625-ccd3-4b10-b370-41ab73e282f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:59:07.305757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-01957625-ccd3-4b10-b370-41ab73e282f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:07.305757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:07.305757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:07.305757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:07.305757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:07.305757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:07.305757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:07.305757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:07.305757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:07.305757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:07.305757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:07.305757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:07.305757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:59:07.305757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:59:07.305757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:59:07.305757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:59:07.307506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:59:07.307506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:59:07.307506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:59:07.307506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:59:07.307506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:07.307506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:07.307506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:07.307506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:07.307506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:07.307506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:07.307506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:07.307506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:07.307506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:07.307506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:07.307506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:07.307506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:07.311382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:07.311382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:07.311382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:07.311382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:07.311382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:59:07.311382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:59:07.311382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:59:07.311382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:59:07.311382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:59:07.311382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:59:07.311382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:07.311382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:07.311382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 271/544] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 14:59:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:07.321829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22b9e6-ae61-4b2a-95c3-f0d1cb7633d1 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:07.407085 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:07.514463 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22b9e6-ae61-4b2a-95c3-f0d1cb7633d1 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:07.566678 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5521e36-fbfd-4289-80af-f23c83216f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:07.567704 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5521e36-fbfd-4289-80af-f23c83216f67 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:07.567704 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5521e36-fbfd-4289-80af-f23c83216f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:07.567982 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5521e36-fbfd-4289-80af-f23c83216f67 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:07.568498 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5521e36-fbfd-4289-80af-f23c83216f67 None admin] RBAC: Adding query filter params (name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:07.571596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5521e36-fbfd-4289-80af-f23c83216f67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:07.571596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5521e36-fbfd-4289-80af-f23c83216f67 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:07.582324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 274/545] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 14:59:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:07.590778 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01957625-ccd3-4b10-b370-41ab73e282f5 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:07.750462 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:07.780057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01957625-ccd3-4b10-b370-41ab73e282f5 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:07.826927 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4664986-8a4c-42ae-a0bb-ed03ce0810da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:07.827648 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4664986-8a4c-42ae-a0bb-ed03ce0810da None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:07.827929 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4664986-8a4c-42ae-a0bb-ed03ce0810da None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:07.828202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4664986-8a4c-42ae-a0bb-ed03ce0810da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:07.840483 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4664986-8a4c-42ae-a0bb-ed03ce0810da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:07.842012 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4664986-8a4c-42ae-a0bb-ed03ce0810da None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:07.865966 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-e4664986-8a4c-42ae-a0bb-ed03ce0810da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:07.865966 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:07.865966 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:07.865966 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:07.865966 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:07.865966 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:07.865966 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:07.865966 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:07.865966 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:07.865966 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:07.865966 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:07.865966 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:07.865966 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:07.865966 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:07.865966 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:07.865966 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:07.867629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:07.867629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:07.867629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:07.867629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:07.867629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:07.867629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:07.867629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:07.867629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:07.867629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:07.867629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:07.867629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:07.867629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:07.867629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:07.867629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:07.867629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:07.867629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:07.867629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:07.869259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:07.869259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:07.869259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:07.869259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 272/546] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:07.875898 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5521e36-fbfd-4289-80af-f23c83216f67 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:08.066738 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:08.084916 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5521e36-fbfd-4289-80af-f23c83216f67 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:08.170924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af0169-7ea8-4325-8a4c-972e81a0cb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:08.171563 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af0169-7ea8-4325-8a4c-972e81a0cb36 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:08.171849 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af0169-7ea8-4325-8a4c-972e81a0cb36 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:08.172128 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af0169-7ea8-4325-8a4c-972e81a0cb36 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:08.172910 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af0169-7ea8-4325-8a4c-972e81a0cb36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:08.173244 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af0169-7ea8-4325-8a4c-972e81a0cb36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:08.174772 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af0169-7ea8-4325-8a4c-972e81a0cb36 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:08.195501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 275/547] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:08.203714 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4664986-8a4c-42ae-a0bb-ed03ce0810da None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:08.277521 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc3f6d-884f-44e1-8103-6e364dfe8841 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:08.277868 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc3f6d-884f-44e1-8103-6e364dfe8841 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:08.278120 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc3f6d-884f-44e1-8103-6e364dfe8841 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:08.279608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 276/548] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:08.294039 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82baae-8ea0-47fd-9e21-7bb5614bac9d None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:08.294484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82baae-8ea0-47fd-9e21-7bb5614bac9d None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:08.294914 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82baae-8ea0-47fd-9e21-7bb5614bac9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:08.347483 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-9e82baae-8ea0-47fd-9e21-7bb5614bac9d None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:08.349531 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-9e82baae-8ea0-47fd-9e21-7bb5614bac9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:08.359448 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4664986-8a4c-42ae-a0bb-ed03ce0810da None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:08.450231 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7e1c1-c131-4c77-a776-f3d5c440bf1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:08.451283 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7e1c1-c131-4c77-a776-f3d5c440bf1c None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:08.452240 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7e1c1-c131-4c77-a776-f3d5c440bf1c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:08.452803 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7e1c1-c131-4c77-a776-f3d5c440bf1c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:08.473000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c7e1c1-c131-4c77-a776-f3d5c440bf1c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:08.474876 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c7e1c1-c131-4c77-a776-f3d5c440bf1c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:08.477245 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:08.497708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-51c7e1c1-c131-4c77-a776-f3d5c440bf1c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 14:59:08.497708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:08.497708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:08.497708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:08.497708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:08.497708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:08.497708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:08.497708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:08.497708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:08.497708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:08.497708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:08.497708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:08.497708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:59:08.497708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:59:08.497708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:59:08.497708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:59:08.499116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:08.499116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:08.499116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:59:08.499116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:08.499116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:59:08.499116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:08.499116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:08.499116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:08.499116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:08.499116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:08.499116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:08.499116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:08.499116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:08.499116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:08.499116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:08.499116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:08.499116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:08.499116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:08.500080 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:08.500080 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:08.500080 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:59:08.500080 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:59:08.500080 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:59:08.500080 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:59:08.500080 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:59:08.500080 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:59:08.500080 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 14:59:08.500080 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:08.501373 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 277/549] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:59:08.509749 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 273/550] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 14:59:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:08.518346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82baae-8ea0-47fd-9e21-7bb5614bac9d None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:08.661801 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:08.675640 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82baae-8ea0-47fd-9e21-7bb5614bac9d None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:08.717115 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6edc53-ff19-4d8e-90af-c7c24883bf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:08.717774 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6edc53-ff19-4d8e-90af-c7c24883bf7b None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:08.718036 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6edc53-ff19-4d8e-90af-c7c24883bf7b None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:08.718294 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6edc53-ff19-4d8e-90af-c7c24883bf7b None admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:08.719044 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6edc53-ff19-4d8e-90af-c7c24883bf7b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:08.719346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6edc53-ff19-4d8e-90af-c7c24883bf7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:08.720815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6edc53-ff19-4d8e-90af-c7c24883bf7b None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:08.740366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 278/551] 10.222.0.44 () {56 vars in 1135 bytes} [Sun Sep 11 14:59:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:08.749031 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7e1c1-c131-4c77-a776-f3d5c440bf1c None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:08.840202 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27a45d-34f4-4e9c-93b8-3e6c3f8c0145 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:08.840673 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27a45d-34f4-4e9c-93b8-3e6c3f8c0145 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:08.841118 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27a45d-34f4-4e9c-93b8-3e6c3f8c0145 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:08.844044 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 279/552] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:59:08.845930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493e694-8356-4d32-8003-083bbff03cc2 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:08.846376 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493e694-8356-4d32-8003-083bbff03cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:08.847177 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493e694-8356-4d32-8003-083bbff03cc2 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:08.849229 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 280/553] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:08.850934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ddb59-74c6-4ec2-8c63-2bc19af08fca None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:08.851381 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ddb59-74c6-4ec2-8c63-2bc19af08fca None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:08.851818 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ddb59-74c6-4ec2-8c63-2bc19af08fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:08.888422 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-ab1ddb59-74c6-4ec2-8c63-2bc19af08fca None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:08.890536 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-ab1ddb59-74c6-4ec2-8c63-2bc19af08fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:08.930136 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7e1c1-c131-4c77-a776-f3d5c440bf1c None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:08.993848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 281/554] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:59:08.994799 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208bff0-a97f-4582-8b86-b1de436e8ccd None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:08.995090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208bff0-a97f-4582-8b86-b1de436e8ccd None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:08.995359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208bff0-a97f-4582-8b86-b1de436e8ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:09.002109 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d4757-467d-4609-94fb-d1b85b1a08f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:09.005070 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d4757-467d-4609-94fb-d1b85b1a08f2 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:09.005756 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d4757-467d-4609-94fb-d1b85b1a08f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:09.006435 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d4757-467d-4609-94fb-d1b85b1a08f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:09.016532 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-2208bff0-a97f-4582-8b86-b1de436e8ccd None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:09.017953 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-2208bff0-a97f-4582-8b86-b1de436e8ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:09.107202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:09.112350 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8d4757-467d-4609-94fb-d1b85b1a08f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:09.114532 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8d4757-467d-4609-94fb-d1b85b1a08f2 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:09.152390 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 282/555] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:59:09.155011 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2208bff0-a97f-4582-8b86-b1de436e8ccd None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:09.161657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:09.295921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 274/556] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:59:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:59:09.303942 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d4757-467d-4609-94fb-d1b85b1a08f2 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:09.368416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2208bff0-a97f-4582-8b86-b1de436e8ccd None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:09.427351 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5dcc6-8b05-481a-b8c2-781b9156d2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:09.428412 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5dcc6-8b05-481a-b8c2-781b9156d2e6 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:09.428868 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5dcc6-8b05-481a-b8c2-781b9156d2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:09.429354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5dcc6-8b05-481a-b8c2-781b9156d2e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:09.450169 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe5dcc6-8b05-481a-b8c2-781b9156d2e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:59:09.460698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-afe5dcc6-8b05-481a-b8c2-781b9156d2e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:09.460698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:09.460698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:09.460698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:09.460698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:09.460698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:09.460698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:09.460698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:09.460698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:09.460698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:09.460698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:09.460698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:09.460698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:59:09.460698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:59:09.460698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:59:09.460698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:59:09.461727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:59:09.461727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:59:09.461727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:59:09.461727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:59:09.461727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:09.461727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:09.461727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:09.461727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:09.461727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:09.461727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:09.461727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:09.461727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:09.461727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:09.461727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:09.461727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:09.461727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:09.462641 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:09.462641 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:09.462641 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:09.462641 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:09.462641 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:59:09.462641 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:59:09.462641 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:59:09.462641 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:59:09.462641 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:59:09.462641 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:59:09.462641 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:09.462641 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:09.463779 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 283/557] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 14:59:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:09.472132 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5dcc6-8b05-481a-b8c2-781b9156d2e6 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:09.492305 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:09.500364 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d4757-467d-4609-94fb-d1b85b1a08f2 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:09.568285 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bafa25-f6a3-497e-80db-38a4137c2f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:09.569205 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bafa25-f6a3-497e-80db-38a4137c2f34 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:09.569817 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bafa25-f6a3-497e-80db-38a4137c2f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:09.570237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bafa25-f6a3-497e-80db-38a4137c2f34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:09.581058 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:09.587285 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bafa25-f6a3-497e-80db-38a4137c2f34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:09.589696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bafa25-f6a3-497e-80db-38a4137c2f34 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:09.610825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-e5bafa25-f6a3-497e-80db-38a4137c2f34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:09.610825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:09.610825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:09.610825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:09.610825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:09.610825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:09.610825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:09.610825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:09.610825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:09.610825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:09.610825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:09.610825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:09.610825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:09.610825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:09.610825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:09.610825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:09.611824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:09.611824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:09.611824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:09.611824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:09.611824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:09.611824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:09.611824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:09.611824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:09.611824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:09.611824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:09.611824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:09.611824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:09.611824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:09.611824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:09.611824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:09.611824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:09.611824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:09.612702 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:09.612702 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:09.612702 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:09.612702 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 275/558] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:09.613443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bafa25-f6a3-497e-80db-38a4137c2f34 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:09.650140 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5dcc6-8b05-481a-b8c2-781b9156d2e6 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:09.692410 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63942063-f4ff-489c-b003-226ec5bd2c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:09.693460 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63942063-f4ff-489c-b003-226ec5bd2c5f None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:09.693460 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63942063-f4ff-489c-b003-226ec5bd2c5f None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:09.693692 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63942063-f4ff-489c-b003-226ec5bd2c5f None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:09.694273 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63942063-f4ff-489c-b003-226ec5bd2c5f None admin] RBAC: Adding query filter params (name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:09.694571 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63942063-f4ff-489c-b003-226ec5bd2c5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:09.695947 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63942063-f4ff-489c-b003-226ec5bd2c5f None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:09.706624 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 284/559] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 14:59:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:59:09.707481 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63942063-f4ff-489c-b003-226ec5bd2c5f None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:09.738625 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bafa25-f6a3-497e-80db-38a4137c2f34 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:09.792639 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d239e0-2516-4dec-8318-552bf6dd484a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:09.793387 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d239e0-2516-4dec-8318-552bf6dd484a None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:09.793656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d239e0-2516-4dec-8318-552bf6dd484a None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:09.793922 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d239e0-2516-4dec-8318-552bf6dd484a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:09.806712 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d239e0-2516-4dec-8318-552bf6dd484a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:59:09.812087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-a0d239e0-2516-4dec-8318-552bf6dd484a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:09.812087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:09.812087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:09.812087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:09.812087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:09.812087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:09.812087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:09.812087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:09.812087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:09.812087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:09.812087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:09.812087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:09.812087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:59:09.812087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:59:09.812087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:59:09.812087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:59:09.813037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:59:09.813037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:59:09.813037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:59:09.813037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:59:09.813037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:09.813037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:09.813037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:09.813037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:09.813037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:09.813037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:09.813037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:09.813037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:09.813037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:09.813037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:09.813037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:09.813037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:09.814081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:09.814081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:09.814081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:09.814081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:09.814081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:59:09.814081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:59:09.814081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:59:09.814081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:59:09.814081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:59:09.814081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:59:09.814081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:09.814081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:09.814081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 276/560] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 14:59:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:09.814806 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d239e0-2516-4dec-8318-552bf6dd484a None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:09.849870 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:09.887277 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63942063-f4ff-489c-b003-226ec5bd2c5f None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:09.919386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:09.935378 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd4bd4-af5b-455b-9e50-b2ebc7da8168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:09.936135 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd4bd4-af5b-455b-9e50-b2ebc7da8168 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:09.936571 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd4bd4-af5b-455b-9e50-b2ebc7da8168 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:09.937009 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd4bd4-af5b-455b-9e50-b2ebc7da8168 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:09.950158 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd4bd4-af5b-455b-9e50-b2ebc7da8168 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:59:09.956268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-afcd4bd4-af5b-455b-9e50-b2ebc7da8168 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:09.956268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:09.956268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:09.956268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:09.956268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:09.956268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:09.956268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:09.956268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:09.956268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:09.956268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:09.956268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:09.956268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:09.956268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:59:09.956268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:59:09.956268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:59:09.956268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:59:09.958306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:59:09.958306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:59:09.958306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:59:09.958306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:59:09.958306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:09.958306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:09.958306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:09.958306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:09.958306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:09.958306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:09.958306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:09.958306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:09.958306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:09.958306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:09.958306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:09.958306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:09.960163 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:09.960163 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:09.960163 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:09.960163 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:09.960163 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:59:09.960163 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:59:09.960163 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:59:09.960163 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:59:09.960163 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:59:09.960163 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:59:09.960163 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:09.960163 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:09.962760 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 285/561] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 14:59:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:09.964037 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd4bd4-af5b-455b-9e50-b2ebc7da8168 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:10.022995 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d239e0-2516-4dec-8318-552bf6dd484a None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:10.074555 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926314f-debb-4009-9224-8facbd8f35b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:10.075174 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926314f-debb-4009-9224-8facbd8f35b3 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:10.076345 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926314f-debb-4009-9224-8facbd8f35b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:10.076610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926314f-debb-4009-9224-8facbd8f35b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:10.079076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926314f-debb-4009-9224-8facbd8f35b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:10.079637 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926314f-debb-4009-9224-8facbd8f35b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:10.080777 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926314f-debb-4009-9224-8facbd8f35b3 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:10.093013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd4bd4-af5b-455b-9e50-b2ebc7da8168 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:10.105892 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 277/562] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:10.106817 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926314f-debb-4009-9224-8facbd8f35b3 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:10.159740 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e001b-3f51-4f0d-b6b7-ce7837443d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:10.160753 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e001b-3f51-4f0d-b6b7-ce7837443d61 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:10.161028 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e001b-3f51-4f0d-b6b7-ce7837443d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:10.161329 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e001b-3f51-4f0d-b6b7-ce7837443d61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:10.175473 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864e001b-3f51-4f0d-b6b7-ce7837443d61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:10.177052 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864e001b-3f51-4f0d-b6b7-ce7837443d61 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:10.193309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-864e001b-3f51-4f0d-b6b7-ce7837443d61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:10.193309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:10.193309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:10.193309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:10.193309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:10.193309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:10.193309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:10.193309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:10.193309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:10.193309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:10.193309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:10.193309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:10.193309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:10.193309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:10.193309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:10.193309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:10.194359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:10.194359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:10.194359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:10.194359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:10.194359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:10.194359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:10.194359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:10.194359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:10.194359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:10.194359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:10.194359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:10.194359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:10.194359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:10.194359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:10.194359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:10.194359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:10.194359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:10.195231 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:10.195231 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:10.195231 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:10.195231 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 286/563] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:10.195231 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e001b-3f51-4f0d-b6b7-ce7837443d61 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:10.240337 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:10.277293 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:10.320866 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926314f-debb-4009-9224-8facbd8f35b3 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:10.328814 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e001b-3f51-4f0d-b6b7-ce7837443d61 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:10.362022 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def2197-f0da-4d5e-99ef-6a1000268525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:10.362537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def2197-f0da-4d5e-99ef-6a1000268525 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:10.362889 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def2197-f0da-4d5e-99ef-6a1000268525 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:10.363086 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def2197-f0da-4d5e-99ef-6a1000268525 None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:10.363764 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def2197-f0da-4d5e-99ef-6a1000268525 None admin] RBAC: Adding query filter params (name=service) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:10.364068 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def2197-f0da-4d5e-99ef-6a1000268525 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:10.365517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def2197-f0da-4d5e-99ef-6a1000268525 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:10.370580 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51965d-4ac3-4f99-81f2-1622886fd64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:10.371328 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51965d-4ac3-4f99-81f2-1622886fd64d None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:10.371605 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51965d-4ac3-4f99-81f2-1622886fd64d None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:10.371867 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51965d-4ac3-4f99-81f2-1622886fd64d None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:10.372547 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51965d-4ac3-4f99-81f2-1622886fd64d None admin] RBAC: Adding query filter params (name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:10.372842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51965d-4ac3-4f99-81f2-1622886fd64d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:10.374402 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51965d-4ac3-4f99-81f2-1622886fd64d None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:10.375011 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 278/564] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 14:59:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:10.375894 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def2197-f0da-4d5e-99ef-6a1000268525 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:10.384476 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 287/565] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 14:59:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:10.385402 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51965d-4ac3-4f99-81f2-1622886fd64d None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:10.485253 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:10.485951 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def2197-f0da-4d5e-99ef-6a1000268525 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:10.489442 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51965d-4ac3-4f99-81f2-1622886fd64d None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:10.517647 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:10.523526 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d9502-e2a8-401f-9dd3-b288e3bfa2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:10.524173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d9502-e2a8-401f-9dd3-b288e3bfa2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:10.524453 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d9502-e2a8-401f-9dd3-b288e3bfa2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:10.524722 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d9502-e2a8-401f-9dd3-b288e3bfa2a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:10.526947 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63b71f-d24e-4101-92c2-7a96dbdd6986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:10.527614 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63b71f-d24e-4101-92c2-7a96dbdd6986 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:10.527889 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63b71f-d24e-4101-92c2-7a96dbdd6986 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:10.528151 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63b71f-d24e-4101-92c2-7a96dbdd6986 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:10.528798 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad63b71f-d24e-4101-92c2-7a96dbdd6986 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:10.529095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad63b71f-d24e-4101-92c2-7a96dbdd6986 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:10.530499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad63b71f-d24e-4101-92c2-7a96dbdd6986 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:10.532322 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d9502-e2a8-401f-9dd3-b288e3bfa2a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:10.533841 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d9502-e2a8-401f-9dd3-b288e3bfa2a9 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:10.548956 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 288/566] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:10.548956 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75466f-4c7e-41a6-9e45-fe04369f2bae None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:10.548956 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75466f-4c7e-41a6-9e45-fe04369f2bae None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:10.548956 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75466f-4c7e-41a6-9e45-fe04369f2bae None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:10.548956 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 289/567] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:10.548956 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75466f-4c7e-41a6-9e45-fe04369f2bae None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:10.551333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-f84d9502-e2a8-401f-9dd3-b288e3bfa2a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:10.551333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:10.551333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:10.551333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:10.551333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:10.551333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:10.551333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:10.551333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:10.551333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:10.551333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:10.551333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:10.551333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:10.551333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:59:10.551333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:59:10.551333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:59:10.552222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:59:10.552222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:10.552222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:10.552222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:10.552222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:10.552222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:10.552222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:10.552222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:10.552222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:10.552222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:10.552222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:10.552222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:10.552222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:10.552222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:10.552222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:10.552222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:10.552222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:10.553101 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:59:10.553101 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:59:10.553101 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:59:10.553101 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:59:10.553101 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:59:10.553101 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:59:10.553101 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:10.553101 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:10.553577 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 279/568] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:59:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:10.554542 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d9502-e2a8-401f-9dd3-b288e3bfa2a9 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:10.665400 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d9502-e2a8-401f-9dd3-b288e3bfa2a9 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:10.665679 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75466f-4c7e-41a6-9e45-fe04369f2bae None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:10.702897 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6729a7-fb89-4279-96c3-0deae380aef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:10.703531 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6729a7-fb89-4279-96c3-0deae380aef0 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:10.703802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6729a7-fb89-4279-96c3-0deae380aef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:10.704065 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6729a7-fb89-4279-96c3-0deae380aef0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:10.704439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc01af-0705-479c-a215-b89232671f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:10.705380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc01af-0705-479c-a215-b89232671f33 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:10.705733 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc01af-0705-479c-a215-b89232671f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:10.706128 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc01af-0705-479c-a215-b89232671f33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:10.715720 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6729a7-fb89-4279-96c3-0deae380aef0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:10.717284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6729a7-fb89-4279-96c3-0deae380aef0 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:10.717481 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecc01af-0705-479c-a215-b89232671f33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:10.718917 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecc01af-0705-479c-a215-b89232671f33 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:10.733018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-8f6729a7-fb89-4279-96c3-0deae380aef0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:10.733018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:10.733018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:10.733018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:10.733018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:10.733018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:10.733018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:10.733018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:10.733018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:10.733018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:10.733018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:10.733018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:10.733018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:10.733018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:10.733018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:10.733018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:10.741981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:10.741981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:10.741981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:10.741981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:10.741981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:10.741981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:10.741981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:10.741981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:10.741981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:10.741981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:10.741981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:10.741981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:10.741981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:10.741981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:10.741981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:10.741981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:10.741981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:10.745035 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:10.745035 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:10.745035 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:10.745035 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 280/569] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:10.745324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-eecc01af-0705-479c-a215-b89232671f33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:10.745324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:10.745324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:10.745324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:10.745324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:10.745324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:10.745324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:10.745324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:10.745324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:10.745324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:10.745324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:10.745324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:10.745324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:10.745324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:10.745324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:10.745324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:10.746324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:10.746324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:10.746324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:10.746324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:10.746324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:10.746324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:10.746324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:10.746324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:10.746324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:10.746324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:10.746324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:10.746324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:10.746324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:10.746324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:10.746324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:10.746324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:10.746324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:10.747404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:10.747404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:10.747404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:10.747583 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6729a7-fb89-4279-96c3-0deae380aef0 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:10.747656 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 290/570] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:10.747656 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc3760-eced-45a6-923c-dca5053f3055 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:10.747656 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc3760-eced-45a6-923c-dca5053f3055 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:10.747656 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc3760-eced-45a6-923c-dca5053f3055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:10.760889 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-d2cc3760-eced-45a6-923c-dca5053f3055 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:10.762242 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-d2cc3760-eced-45a6-923c-dca5053f3055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:10.765575 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:10.790346 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:10.853239 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 291/571] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:59:10.853239 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc3760-eced-45a6-923c-dca5053f3055 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:10.853839 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6729a7-fb89-4279-96c3-0deae380aef0 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:10.890651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fda59b-aaae-4b12-91a7-74ce16968238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:10.891271 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fda59b-aaae-4b12-91a7-74ce16968238 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:10.891547 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fda59b-aaae-4b12-91a7-74ce16968238 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:10.891809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fda59b-aaae-4b12-91a7-74ce16968238 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:10.899996 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fda59b-aaae-4b12-91a7-74ce16968238 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:10.901493 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fda59b-aaae-4b12-91a7-74ce16968238 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:10.913321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-02fda59b-aaae-4b12-91a7-74ce16968238 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 11 14:59:10.913321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:10.913321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:10.913321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:10.913321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:10.913321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:10.913321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:10.913321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:10.913321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:10.913321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:10.913321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:10.913321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:10.913321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:59:10.913321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:59:10.913321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:59:10.913321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:59:10.919654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:10.919654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:10.919654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:59:10.919654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:10.919654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:59:10.919654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:10.919654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:10.919654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:10.919654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:10.919654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:10.919654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:10.919654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:10.919654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:10.919654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:10.919654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:10.919654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:10.919654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:10.919654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:10.920687 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:10.920687 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:10.920687 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:59:10.920687 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:59:10.920687 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:59:10.920687 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:59:10.920687 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:59:10.920687 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:59:10.920687 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 11 14:59:10.920687 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:10.920687 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 281/572] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:59:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:10.920687 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fda59b-aaae-4b12-91a7-74ce16968238 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:11.010041 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc3760-eced-45a6-923c-dca5053f3055 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:11.040662 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fda59b-aaae-4b12-91a7-74ce16968238 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:11.049889 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:11.052026 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f17150-a1df-4e54-8b2a-6925703031a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:11.052664 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f17150-a1df-4e54-8b2a-6925703031a4 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:11.052931 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f17150-a1df-4e54-8b2a-6925703031a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:11.053234 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f17150-a1df-4e54-8b2a-6925703031a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:11.053923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f17150-a1df-4e54-8b2a-6925703031a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:11.054225 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f17150-a1df-4e54-8b2a-6925703031a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:11.055897 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f17150-a1df-4e54-8b2a-6925703031a4 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:11.066583 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:11.078341 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 292/573] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 14:59:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:11.078341 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f17150-a1df-4e54-8b2a-6925703031a4 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:11.096461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b5c5e-5003-406d-9362-08bf79f96d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:11.097468 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b5c5e-5003-406d-9362-08bf79f96d83 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:11.097995 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b5c5e-5003-406d-9362-08bf79f96d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:11.098234 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b5c5e-5003-406d-9362-08bf79f96d83 None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:11.099237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b5c5e-5003-406d-9362-08bf79f96d83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:11.099905 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b5c5e-5003-406d-9362-08bf79f96d83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:11.101527 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b5c5e-5003-406d-9362-08bf79f96d83 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:11.117001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 282/574] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:11.117909 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b5c5e-5003-406d-9362-08bf79f96d83 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:11.207281 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f17150-a1df-4e54-8b2a-6925703031a4 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:11.233577 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b5c5e-5003-406d-9362-08bf79f96d83 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:11.262544 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3e328-dd2d-4e1a-aa51-14ccf67b6eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:11.263129 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3e328-dd2d-4e1a-aa51-14ccf67b6eaf None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:11.263397 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3e328-dd2d-4e1a-aa51-14ccf67b6eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:11.263661 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3e328-dd2d-4e1a-aa51-14ccf67b6eaf None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:11.264309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3e328-dd2d-4e1a-aa51-14ccf67b6eaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:11.264603 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3e328-dd2d-4e1a-aa51-14ccf67b6eaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:11.266021 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3e328-dd2d-4e1a-aa51-14ccf67b6eaf None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:11.278924 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73646304-2c18-460f-950b-87215d15b111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:11.279161 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 293/575] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:11.279296 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73646304-2c18-460f-950b-87215d15b111 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:11.279525 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73646304-2c18-460f-950b-87215d15b111 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:11.279673 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3e328-dd2d-4e1a-aa51-14ccf67b6eaf None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:11.279833 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73646304-2c18-460f-950b-87215d15b111 None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:11.280569 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73646304-2c18-460f-950b-87215d15b111 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:11.280875 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73646304-2c18-460f-950b-87215d15b111 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:11.282389 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73646304-2c18-460f-950b-87215d15b111 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:11.303782 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 283/576] 10.222.0.44 () {56 vars in 1137 bytes} [Sun Sep 11 14:59:11 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:11.304791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73646304-2c18-460f-950b-87215d15b111 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:11.334325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:11.346443 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:11.390524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3e328-dd2d-4e1a-aa51-14ccf67b6eaf None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:11.415243 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73646304-2c18-460f-950b-87215d15b111 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:11.427612 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0202cfc4-bf6a-49c1-8896-382ae2cd56db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:11.428290 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0202cfc4-bf6a-49c1-8896-382ae2cd56db None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:11.428626 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0202cfc4-bf6a-49c1-8896-382ae2cd56db None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:11.428965 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0202cfc4-bf6a-49c1-8896-382ae2cd56db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:11.429789 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0202cfc4-bf6a-49c1-8896-382ae2cd56db None admin] RBAC: Adding query filter params (role.id=ea3b2f1eb283418b8823cb4fff80edba, scope.project.id=70ac515938a24c45ae6302dbac46a1ab, user.id=05aa5e40ee4542719657bc03b2015e72) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:11.430161 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0202cfc4-bf6a-49c1-8896-382ae2cd56db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:11.431678 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0202cfc4-bf6a-49c1-8896-382ae2cd56db None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:11.438240 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 294/577] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:59:11 2022] GET /identity/v3/role_assignments?user.id=05aa5e40ee4542719657bc03b2015e72&scope.project.id=70ac515938a24c45ae6302dbac46a1ab&role.id=ea3b2f1eb283418b8823cb4fff80edba => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:11.439688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0202cfc4-bf6a-49c1-8896-382ae2cd56db None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:11.471972 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420fcc6-ae30-4785-b2d2-77010216a8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:11.472677 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420fcc6-ae30-4785-b2d2-77010216a8e0 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:11.472926 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420fcc6-ae30-4785-b2d2-77010216a8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:11.473278 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420fcc6-ae30-4785-b2d2-77010216a8e0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:11.483293 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0420fcc6-ae30-4785-b2d2-77010216a8e0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:11.484849 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0420fcc6-ae30-4785-b2d2-77010216a8e0 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:11.499124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-0420fcc6-ae30-4785-b2d2-77010216a8e0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 14:59:11.499124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:11.499124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:11.499124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:11.499124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:11.499124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:11.499124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:11.499124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:11.499124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:11.499124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:11.499124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:11.499124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:11.499124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:59:11.499124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:59:11.499124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:59:11.499124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:59:11.500975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:11.500975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:11.500975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:59:11.500975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:11.500975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:59:11.500975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:11.500975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:11.500975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:11.500975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:11.500975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:11.500975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:11.500975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:11.500975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:11.500975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:11.500975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:11.500975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:11.500975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:11.500975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:11.502851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:11.502851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:11.502851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:59:11.502851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:59:11.502851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:59:11.502851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:59:11.502851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:59:11.502851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:59:11.502851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 14:59:11.502851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:11.505889 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 284/578] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 14:59:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:11.507749 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420fcc6-ae30-4785-b2d2-77010216a8e0 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:11.601961 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0202cfc4-bf6a-49c1-8896-382ae2cd56db None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:11.642308 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:11.649480 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:11.657095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee725862-2bd2-421e-a83e-eb46e9adf228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:11.658467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee725862-2bd2-421e-a83e-eb46e9adf228 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:11.658747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee725862-2bd2-421e-a83e-eb46e9adf228 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:11.659008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee725862-2bd2-421e-a83e-eb46e9adf228 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:11.672583 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420fcc6-ae30-4785-b2d2-77010216a8e0 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:11.673074 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee725862-2bd2-421e-a83e-eb46e9adf228 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:59:11.678610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-ee725862-2bd2-421e-a83e-eb46e9adf228 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:11.678610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:11.678610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:11.678610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:11.678610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:11.678610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:11.678610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:11.678610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:11.678610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:11.678610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:11.678610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:11.678610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:11.678610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:59:11.678610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:59:11.678610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:59:11.678610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:59:11.685812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:59:11.685812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:59:11.685812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:59:11.685812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:59:11.685812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:11.685812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:11.685812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:11.685812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:11.685812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:11.685812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:11.685812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:11.685812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:11.685812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:11.685812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:11.685812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:11.685812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:11.686871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:11.686871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:11.686871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:11.686871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:11.686871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:59:11.686871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:59:11.686871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:59:11.686871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:59:11.686871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:59:11.686871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:59:11.686871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:11.686871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:11.686871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 295/579] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 14:59:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:11.686871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee725862-2bd2-421e-a83e-eb46e9adf228 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:11.736434 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd7b1db-3652-4ced-a99d-d508ea1d0fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:11.737557 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd7b1db-3652-4ced-a99d-d508ea1d0fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:11.738029 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd7b1db-3652-4ced-a99d-d508ea1d0fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:11.738500 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd7b1db-3652-4ced-a99d-d508ea1d0fa8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:11.752681 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd7b1db-3652-4ced-a99d-d508ea1d0fa8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:11.755340 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd7b1db-3652-4ced-a99d-d508ea1d0fa8 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:11.776978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-dcd7b1db-3652-4ced-a99d-d508ea1d0fa8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:59:11.776978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:11.776978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:11.776978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:11.776978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:11.776978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:11.776978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:11.776978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:11.776978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:11.776978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:11.776978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:11.776978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:11.776978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:59:11.776978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:59:11.776978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:59:11.776978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:59:11.778056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:11.778056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:11.778056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:59:11.778056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:11.778056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:59:11.778056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:11.778056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:11.778056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:11.778056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:11.778056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:11.778056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:11.778056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:11.778056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:11.778056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:11.778056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:11.778056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:11.778056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:11.778056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:11.779051 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:11.779051 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:11.779051 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:59:11.779051 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:59:11.779051 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:59:11.779051 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:59:11.779051 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:59:11.779051 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:59:11.779051 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:59:11.779051 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:11.786382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 285/580] 10.222.0.44 () {56 vars in 1093 bytes} [Sun Sep 11 14:59:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:11.787834 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd7b1db-3652-4ced-a99d-d508ea1d0fa8 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:11.808861 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee725862-2bd2-421e-a83e-eb46e9adf228 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:11.867929 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91c780-1f47-4ea9-ad05-c60b56755346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:11.869064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91c780-1f47-4ea9-ad05-c60b56755346 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:11.869644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91c780-1f47-4ea9-ad05-c60b56755346 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:11.870179 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91c780-1f47-4ea9-ad05-c60b56755346 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:11.890606 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c91c780-1f47-4ea9-ad05-c60b56755346 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:11.893520 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c91c780-1f47-4ea9-ad05-c60b56755346 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:11.918976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-8c91c780-1f47-4ea9-ad05-c60b56755346 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:11.918976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:11.918976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:11.918976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:11.918976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:11.918976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:11.918976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:11.918976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:11.918976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:11.918976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:11.918976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:11.918976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:11.918976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:11.918976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:11.918976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:11.918976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:11.920079 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:11.920079 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:11.920079 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:11.920079 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:11.920079 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:11.920079 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:11.920079 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:11.920079 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:11.920079 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:11.920079 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:11.920079 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:11.920079 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:11.920079 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:11.920079 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:11.920079 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:11.920079 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:11.920079 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:11.921075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:11.921075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:11.921075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:11.922519 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 296/581] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:11.923387 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91c780-1f47-4ea9-ad05-c60b56755346 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:11.965828 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd7b1db-3652-4ced-a99d-d508ea1d0fa8 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:11.970543 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:12.006810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58703be-a661-4926-841f-7598e6fbe2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:12.007345 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58703be-a661-4926-841f-7598e6fbe2aa None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:12.008019 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58703be-a661-4926-841f-7598e6fbe2aa None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:12.008019 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58703be-a661-4926-841f-7598e6fbe2aa None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:12.008726 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58703be-a661-4926-841f-7598e6fbe2aa None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:12.009036 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58703be-a661-4926-841f-7598e6fbe2aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:12.010565 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58703be-a661-4926-841f-7598e6fbe2aa None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:12.036107 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:12.042680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 286/582] 10.222.0.44 () {56 vars in 1139 bytes} [Sun Sep 11 14:59:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:12.044217 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58703be-a661-4926-841f-7598e6fbe2aa None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:12.062012 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91c780-1f47-4ea9-ad05-c60b56755346 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:12.114784 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6c7a9-b7c0-497c-ade8-8c0ebd38ad32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:12.115740 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6c7a9-b7c0-497c-ade8-8c0ebd38ad32 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:12.116186 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6c7a9-b7c0-497c-ade8-8c0ebd38ad32 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:12.116632 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6c7a9-b7c0-497c-ade8-8c0ebd38ad32 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:12.117832 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6c7a9-b7c0-497c-ade8-8c0ebd38ad32 None admin] RBAC: Adding query filter params (name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:12.118330 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6c7a9-b7c0-497c-ade8-8c0ebd38ad32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:12.120781 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6c7a9-b7c0-497c-ade8-8c0ebd38ad32 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:12.140200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 297/583] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 14:59:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:12.140200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6c7a9-b7c0-497c-ade8-8c0ebd38ad32 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:12.225789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58703be-a661-4926-841f-7598e6fbe2aa None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:12.268890 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f11d74-3497-4206-bd2b-9d5817a6a051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:12.269690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f11d74-3497-4206-bd2b-9d5817a6a051 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:12.269972 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f11d74-3497-4206-bd2b-9d5817a6a051 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:12.270306 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f11d74-3497-4206-bd2b-9d5817a6a051 None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:12.273847 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f11d74-3497-4206-bd2b-9d5817a6a051 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:12.273847 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f11d74-3497-4206-bd2b-9d5817a6a051 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:12.273847 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f11d74-3497-4206-bd2b-9d5817a6a051 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:12.305804 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:12.306176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 287/584] 10.222.0.44 () {56 vars in 1133 bytes} [Sun Sep 11 14:59:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:12.310967 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6c7a9-b7c0-497c-ade8-8c0ebd38ad32 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:12.314206 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f11d74-3497-4206-bd2b-9d5817a6a051 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:12.357282 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ffea4-d06a-4405-9cbd-7f843ae963a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:12.358165 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ffea4-d06a-4405-9cbd-7f843ae963a1 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:12.358610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ffea4-d06a-4405-9cbd-7f843ae963a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:12.359038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ffea4-d06a-4405-9cbd-7f843ae963a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:12.359694 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ffea4-d06a-4405-9cbd-7f843ae963a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:12.359992 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ffea4-d06a-4405-9cbd-7f843ae963a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:12.361425 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ffea4-d06a-4405-9cbd-7f843ae963a1 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:12.375784 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 298/585] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:12.376709 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ffea4-d06a-4405-9cbd-7f843ae963a1 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:12.392709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:12.438946 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f11d74-3497-4206-bd2b-9d5817a6a051 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:12.477576 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109ffc6-3baa-4c48-a8b8-901a3f9514dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:12.478424 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109ffc6-3baa-4c48-a8b8-901a3f9514dc None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:12.478830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109ffc6-3baa-4c48-a8b8-901a3f9514dc None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:12.479273 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109ffc6-3baa-4c48-a8b8-901a3f9514dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:12.492512 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c109ffc6-3baa-4c48-a8b8-901a3f9514dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:12.494307 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c109ffc6-3baa-4c48-a8b8-901a3f9514dc None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:12.512629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-c109ffc6-3baa-4c48-a8b8-901a3f9514dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:12.512629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:12.512629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:12.512629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:12.512629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:12.512629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:12.512629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:12.512629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:12.512629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:12.512629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:12.512629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:12.512629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:12.512629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:12.512629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:12.512629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:12.512629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:12.514537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:12.514537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:12.514537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:12.514537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:12.514537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:12.514537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:12.514537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:12.514537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:12.514537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:12.514537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:12.514537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:12.514537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:12.514537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:12.514537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:12.514537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:12.514537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:12.514537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:12.517112 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:12.517112 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:12.517112 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:12.517112 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 288/586] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:12.517112 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109ffc6-3baa-4c48-a8b8-901a3f9514dc None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:12.553613 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ffea4-d06a-4405-9cbd-7f843ae963a1 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:12.600435 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06606602-7164-49a2-8c29-2782e153685d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:12.601084 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06606602-7164-49a2-8c29-2782e153685d None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:12.601397 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06606602-7164-49a2-8c29-2782e153685d None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:12.601674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06606602-7164-49a2-8c29-2782e153685d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:12.613705 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06606602-7164-49a2-8c29-2782e153685d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:12.615172 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06606602-7164-49a2-8c29-2782e153685d None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:12.632540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-06606602-7164-49a2-8c29-2782e153685d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:12.632540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:12.632540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:12.632540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:12.632540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:12.632540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:12.632540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:12.632540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:12.632540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:12.632540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:12.632540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:12.632540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:12.632540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:12.632540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:12.632540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:12.632540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:12.634386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:12.634386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:12.634386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:12.634386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:12.634386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:12.634386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:12.634386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:12.634386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:12.634386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:12.634386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:12.634386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:12.634386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:12.634386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:12.634386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:12.634386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:12.634386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:12.634386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:12.636091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:12.636091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:12.636091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:12.636091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 299/587] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:12.636091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06606602-7164-49a2-8c29-2782e153685d None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:12.649654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109ffc6-3baa-4c48-a8b8-901a3f9514dc None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:12.652822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:12.663637 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:12.690664 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31412c1-7f80-4f81-a0da-b127e346a4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:12.691553 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31412c1-7f80-4f81-a0da-b127e346a4b1 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:12.691807 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31412c1-7f80-4f81-a0da-b127e346a4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:12.692083 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31412c1-7f80-4f81-a0da-b127e346a4b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:12.703772 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31412c1-7f80-4f81-a0da-b127e346a4b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:12.705283 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31412c1-7f80-4f81-a0da-b127e346a4b1 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:12.720698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-d31412c1-7f80-4f81-a0da-b127e346a4b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:12.720698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:12.720698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:12.720698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:12.720698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:12.720698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:12.720698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:12.720698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:12.720698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:12.720698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:12.720698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:12.720698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:12.720698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:12.720698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:12.720698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:12.720698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:12.722173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:12.722173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:12.722173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:12.722173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:12.722173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:12.722173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:12.722173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:12.722173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:12.722173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:12.722173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:12.722173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:12.722173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:12.722173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:12.722173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:12.722173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:12.722173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:12.722173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:12.723160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:12.723160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:12.723160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:12.727089 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 289/588] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:12.728005 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31412c1-7f80-4f81-a0da-b127e346a4b1 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:12.773501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06606602-7164-49a2-8c29-2782e153685d None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:12.821666 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444c518-f9c0-41fd-be8e-88f1e0264f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:12.823145 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444c518-f9c0-41fd-be8e-88f1e0264f74 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:12.823902 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444c518-f9c0-41fd-be8e-88f1e0264f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:12.824409 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444c518-f9c0-41fd-be8e-88f1e0264f74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:12.834081 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444c518-f9c0-41fd-be8e-88f1e0264f74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:12.835980 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444c518-f9c0-41fd-be8e-88f1e0264f74 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:12.844605 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31412c1-7f80-4f81-a0da-b127e346a4b1 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:12.853940 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-0444c518-f9c0-41fd-be8e-88f1e0264f74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:12.853940 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:12.853940 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:12.853940 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:12.853940 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:12.853940 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:12.853940 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:12.853940 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:12.853940 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:12.853940 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:12.853940 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:12.853940 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:12.853940 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:59:12.853940 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:59:12.853940 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:59:12.855644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:59:12.855644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:12.855644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:12.855644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:12.855644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:12.855644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:12.855644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:12.855644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:12.855644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:12.855644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:12.855644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:12.855644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:12.855644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:12.855644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:12.855644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:12.855644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:12.855644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:12.857447 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:59:12.857447 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:59:12.857447 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:59:12.857447 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:59:12.857447 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:59:12.857447 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:59:12.857447 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:12.857447 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:12.857447 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 300/589] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:59:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:12.857447 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444c518-f9c0-41fd-be8e-88f1e0264f74 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:12.901935 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c0b00-e18e-457b-84b6-d7a6e9397b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:12.903012 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c0b00-e18e-457b-84b6-d7a6e9397b8f None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:12.903374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c0b00-e18e-457b-84b6-d7a6e9397b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:12.903757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c0b00-e18e-457b-84b6-d7a6e9397b8f None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:12.904525 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c0b00-e18e-457b-84b6-d7a6e9397b8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:12.904914 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c0b00-e18e-457b-84b6-d7a6e9397b8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:12.907216 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c0b00-e18e-457b-84b6-d7a6e9397b8f None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:12.933872 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 290/590] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:12.933872 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c0b00-e18e-457b-84b6-d7a6e9397b8f None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:12.961816 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:12.969423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:13.010148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444c518-f9c0-41fd-be8e-88f1e0264f74 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:13.055359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fc52f-8d54-4814-a0a4-845ca2a7f6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:13.056351 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fc52f-8d54-4814-a0a4-845ca2a7f6e0 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:13.056802 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fc52f-8d54-4814-a0a4-845ca2a7f6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:13.057222 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fc52f-8d54-4814-a0a4-845ca2a7f6e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:13.058340 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386fc52f-8d54-4814-a0a4-845ca2a7f6e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:13.058818 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386fc52f-8d54-4814-a0a4-845ca2a7f6e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:13.061028 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386fc52f-8d54-4814-a0a4-845ca2a7f6e0 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:13.070054 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c0b00-e18e-457b-84b6-d7a6e9397b8f None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:13.086423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 301/591] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:13.086423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fc52f-8d54-4814-a0a4-845ca2a7f6e0 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:13.132341 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f89786-1116-4f6a-b324-df4726ed1fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:13.133319 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f89786-1116-4f6a-b324-df4726ed1fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:13.133780 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f89786-1116-4f6a-b324-df4726ed1fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:13.134019 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f89786-1116-4f6a-b324-df4726ed1fe6 None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:13.135001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f89786-1116-4f6a-b324-df4726ed1fe6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:13.135708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f89786-1116-4f6a-b324-df4726ed1fe6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:13.137565 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f89786-1116-4f6a-b324-df4726ed1fe6 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:13.158310 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 291/592] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:13.158310 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f89786-1116-4f6a-b324-df4726ed1fe6 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:13.266798 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fc52f-8d54-4814-a0a4-845ca2a7f6e0 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:13.282550 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:13.304483 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f89786-1116-4f6a-b324-df4726ed1fe6 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:13.323708 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:13.336169 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56dbc3-e088-4d17-8d2b-3f7af830f5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:13.337568 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56dbc3-e088-4d17-8d2b-3f7af830f5c3 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:13.337913 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56dbc3-e088-4d17-8d2b-3f7af830f5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:13.338344 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56dbc3-e088-4d17-8d2b-3f7af830f5c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:13.339165 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c56dbc3-e088-4d17-8d2b-3f7af830f5c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:13.339475 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c56dbc3-e088-4d17-8d2b-3f7af830f5c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:13.341046 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c56dbc3-e088-4d17-8d2b-3f7af830f5c3 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:13.353914 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a95a03-12c2-4874-b3fe-6bfc11238dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:13.361214 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a95a03-12c2-4874-b3fe-6bfc11238dab None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:13.361421 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 302/593] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 14:59:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:13.361537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a95a03-12c2-4874-b3fe-6bfc11238dab None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:13.361537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a95a03-12c2-4874-b3fe-6bfc11238dab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:13.361719 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56dbc3-e088-4d17-8d2b-3f7af830f5c3 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:13.370931 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a95a03-12c2-4874-b3fe-6bfc11238dab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:13.373887 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a95a03-12c2-4874-b3fe-6bfc11238dab None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:13.395668 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-71a95a03-12c2-4874-b3fe-6bfc11238dab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:13.395668 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:13.395668 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:13.395668 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:13.395668 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:13.395668 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:13.395668 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:13.395668 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:13.395668 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:13.395668 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:13.395668 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:13.395668 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:13.395668 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:59:13.395668 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:59:13.395668 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:59:13.396827 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:59:13.396827 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:13.396827 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:13.396827 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:13.396827 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:13.396827 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:13.396827 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:13.396827 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:13.396827 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:13.396827 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:13.396827 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:13.396827 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:13.396827 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:13.396827 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:13.396827 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:13.396827 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:13.396827 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:13.397964 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:59:13.397964 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:59:13.397964 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:59:13.397964 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:59:13.397964 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:59:13.397964 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:59:13.397964 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:13.397964 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:13.400084 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 292/594] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:59:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:13.401646 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a95a03-12c2-4874-b3fe-6bfc11238dab None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:13.525692 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56dbc3-e088-4d17-8d2b-3f7af830f5c3 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:13.559203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a95a03-12c2-4874-b3fe-6bfc11238dab None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:13.600990 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4180c96-da43-4eaf-8eaa-07d83f3429a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:13.602130 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4180c96-da43-4eaf-8eaa-07d83f3429a0 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:13.602771 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4180c96-da43-4eaf-8eaa-07d83f3429a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:13.602965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ced5e-3608-4735-8bf3-38fbe894be8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:13.603696 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4180c96-da43-4eaf-8eaa-07d83f3429a0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:13.603867 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ced5e-3608-4735-8bf3-38fbe894be8a None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:13.604186 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ced5e-3608-4735-8bf3-38fbe894be8a None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:13.604604 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ced5e-3608-4735-8bf3-38fbe894be8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:13.614024 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ced5e-3608-4735-8bf3-38fbe894be8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:13.615527 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ced5e-3608-4735-8bf3-38fbe894be8a None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:13.619482 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4180c96-da43-4eaf-8eaa-07d83f3429a0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:13.622054 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4180c96-da43-4eaf-8eaa-07d83f3429a0 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:13.629492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-6b4ced5e-3608-4735-8bf3-38fbe894be8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:13.629492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:13.629492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:13.629492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:13.629492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:13.629492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:13.629492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:13.629492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:13.629492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:13.629492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:13.629492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:13.629492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:13.629492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:59:13.629492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:59:13.629492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:59:13.630717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:59:13.630717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:13.630717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:13.630717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:13.630717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:13.630717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:13.630717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:13.630717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:13.630717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:13.630717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:13.630717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:13.630717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:13.630717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:13.630717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:13.630717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:13.630717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:13.630717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:13.632076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:59:13.632076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:59:13.632076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:59:13.632076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:59:13.632076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:59:13.632076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:59:13.632076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:13.632076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:13.632076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 293/595] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:59:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:13.632076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ced5e-3608-4735-8bf3-38fbe894be8a None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:13.643072 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:13.648508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-b4180c96-da43-4eaf-8eaa-07d83f3429a0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 14:59:13.648508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:13.648508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:13.648508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:13.648508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:13.648508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:13.648508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:13.648508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:13.648508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:13.648508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:13.648508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:13.648508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:13.648508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:59:13.648508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:59:13.648508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:59:13.648508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:59:13.650358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:13.650358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:13.650358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:59:13.650358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:13.650358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:59:13.650358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:13.650358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:13.650358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:13.650358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:13.650358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:13.650358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:13.650358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:13.650358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:13.650358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:13.650358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:13.650358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:13.650358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:13.650358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:13.652456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:13.652456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:13.652456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:59:13.652456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:59:13.652456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:59:13.652456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:59:13.652456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:59:13.652456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:59:13.652456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 14:59:13.652456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:13.653566 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 303/596] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:59:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:13.654784 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4180c96-da43-4eaf-8eaa-07d83f3429a0 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:13.739591 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:13.808393 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4180c96-da43-4eaf-8eaa-07d83f3429a0 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:13.824565 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ced5e-3608-4735-8bf3-38fbe894be8a None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:13.859789 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a049e57-da55-4e1c-a2f3-8d5f39b225d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:13.860737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a049e57-da55-4e1c-a2f3-8d5f39b225d7 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:13.861570 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a049e57-da55-4e1c-a2f3-8d5f39b225d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:13.862055 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a049e57-da55-4e1c-a2f3-8d5f39b225d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:13.863314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a049e57-da55-4e1c-a2f3-8d5f39b225d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:13.863865 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a049e57-da55-4e1c-a2f3-8d5f39b225d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:13.866773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a049e57-da55-4e1c-a2f3-8d5f39b225d7 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:13.884441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3aa3fd-c669-4fd1-b3c9-0fa2fc5ff047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:13.885178 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3aa3fd-c669-4fd1-b3c9-0fa2fc5ff047 None admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:13.885449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3aa3fd-c669-4fd1-b3c9-0fa2fc5ff047 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:13.885712 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3aa3fd-c669-4fd1-b3c9-0fa2fc5ff047 None admin] PATH_INFO: `/v3/projects/70ac515938a24c45ae6302dbac46a1ab/users/a1fe5293447346eaafd16818e5bd12f4/roles/ea3b2f1eb283418b8823cb4fff80edba` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:13.891284 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 304/597] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 14:59:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:13.893183 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a049e57-da55-4e1c-a2f3-8d5f39b225d7 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:13.902732 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3aa3fd-c669-4fd1-b3c9-0fa2fc5ff047 None admin] RBAC: Authorizing `identity:create_grant(project_id=70ac515938a24c45ae6302dbac46a1ab, user_id=a1fe5293447346eaafd16818e5bd12f4, role_id=ea3b2f1eb283418b8823cb4fff80edba)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:13.904209 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3aa3fd-c669-4fd1-b3c9-0fa2fc5ff047 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:13.922575 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 294/598] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 14:59:13 2022] PUT /identity/v3/projects/70ac515938a24c45ae6302dbac46a1ab/users/a1fe5293447346eaafd16818e5bd12f4/roles/ea3b2f1eb283418b8823cb4fff80edba => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:59:13.924514 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3aa3fd-c669-4fd1-b3c9-0fa2fc5ff047 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:14.009018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:14.051273 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a049e57-da55-4e1c-a2f3-8d5f39b225d7 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:14.065724 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:14.071245 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3aa3fd-c669-4fd1-b3c9-0fa2fc5ff047 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:14.098789 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aba0ee-0271-445b-9e9d-6a72834d81bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:14.099404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aba0ee-0271-445b-9e9d-6a72834d81bc None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:14.099667 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aba0ee-0271-445b-9e9d-6a72834d81bc None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:14.099939 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aba0ee-0271-445b-9e9d-6a72834d81bc None admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:14.100828 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aba0ee-0271-445b-9e9d-6a72834d81bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:14.101127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aba0ee-0271-445b-9e9d-6a72834d81bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:14.102612 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aba0ee-0271-445b-9e9d-6a72834d81bc None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:14.115544 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 305/599] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 14:59:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:14.116367 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aba0ee-0271-445b-9e9d-6a72834d81bc None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:14.126528 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480697b-c583-4ade-abe8-96c984ad0ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:14.127168 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480697b-c583-4ade-abe8-96c984ad0ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:14.127445 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480697b-c583-4ade-abe8-96c984ad0ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:14.127795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480697b-c583-4ade-abe8-96c984ad0ec8 None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:14.128833 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480697b-c583-4ade-abe8-96c984ad0ec8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:14.129168 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480697b-c583-4ade-abe8-96c984ad0ec8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:14.130722 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480697b-c583-4ade-abe8-96c984ad0ec8 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:14.152845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 295/600] 10.222.0.44 () {56 vars in 1137 bytes} [Sun Sep 11 14:59:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:14.159010 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f860a-40f8-4b82-95e8-d1950acf284f None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:14.159340 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f860a-40f8-4b82-95e8-d1950acf284f None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:14.159678 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f860a-40f8-4b82-95e8-d1950acf284f None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:14.161122 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 296/601] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:14.163012 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f860a-40f8-4b82-95e8-d1950acf284f None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:14.291824 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aba0ee-0271-445b-9e9d-6a72834d81bc None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:14.334148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2353c3-1807-43cb-a7e6-3c81ebc0a3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:14.334915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2353c3-1807-43cb-a7e6-3c81ebc0a3a6 None admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:14.335345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2353c3-1807-43cb-a7e6-3c81ebc0a3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:14.335709 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2353c3-1807-43cb-a7e6-3c81ebc0a3a6 None admin] PATH_INFO: `/v3/projects/70ac515938a24c45ae6302dbac46a1ab/users/697b73f182e6419a84e0c8c6b0fc0c06/roles/ea3b2f1eb283418b8823cb4fff80edba` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:14.361819 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2353c3-1807-43cb-a7e6-3c81ebc0a3a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=70ac515938a24c45ae6302dbac46a1ab, user_id=697b73f182e6419a84e0c8c6b0fc0c06, role_id=ea3b2f1eb283418b8823cb4fff80edba)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:14.363336 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2353c3-1807-43cb-a7e6-3c81ebc0a3a6 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:14.364911 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f860a-40f8-4b82-95e8-d1950acf284f None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:14.388553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 306/602] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 14:59:14 2022] PUT /identity/v3/projects/70ac515938a24c45ae6302dbac46a1ab/users/697b73f182e6419a84e0c8c6b0fc0c06/roles/ea3b2f1eb283418b8823cb4fff80edba => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:59:14.388553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2353c3-1807-43cb-a7e6-3c81ebc0a3a6 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:14.388933 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:14.421864 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:14.438883 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8862101c-40de-4f10-ac99-2628797246e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:14.441399 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862101c-40de-4f10-ac99-2628797246e3 None admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:14.442603 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862101c-40de-4f10-ac99-2628797246e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:14.444293 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862101c-40de-4f10-ac99-2628797246e3 None admin] PATH_INFO: `/v3/projects/70ac515938a24c45ae6302dbac46a1ab/users/453297dac64046d19fc1c331d2d85abc/roles/ea3b2f1eb283418b8823cb4fff80edba` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:14.469795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8862101c-40de-4f10-ac99-2628797246e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=70ac515938a24c45ae6302dbac46a1ab, user_id=453297dac64046d19fc1c331d2d85abc, role_id=ea3b2f1eb283418b8823cb4fff80edba)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:14.472077 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8862101c-40de-4f10-ac99-2628797246e3 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:14.498626 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 297/603] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 14:59:14 2022] PUT /identity/v3/projects/70ac515938a24c45ae6302dbac46a1ab/users/453297dac64046d19fc1c331d2d85abc/roles/ea3b2f1eb283418b8823cb4fff80edba => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:59:14.498626 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5139d-537d-4a61-be71-c23c955d835c None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:14.498626 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5139d-537d-4a61-be71-c23c955d835c None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:14.498626 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5139d-537d-4a61-be71-c23c955d835c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:14.525183 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-19d5139d-537d-4a61-be71-c23c955d835c None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:14.525183 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-19d5139d-537d-4a61-be71-c23c955d835c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:14.622673 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2353c3-1807-43cb-a7e6-3c81ebc0a3a6 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:14.646871 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 298/604] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:59:14.669343 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c37c4-538a-4f68-a698-15c5ae719905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:14.670121 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c37c4-538a-4f68-a698-15c5ae719905 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:14.670400 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c37c4-538a-4f68-a698-15c5ae719905 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:14.670673 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c37c4-538a-4f68-a698-15c5ae719905 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:14.685885 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7c37c4-538a-4f68-a698-15c5ae719905 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:14.687500 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7c37c4-538a-4f68-a698-15c5ae719905 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:14.705407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-ae7c37c4-538a-4f68-a698-15c5ae719905 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:14.705407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:14.705407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:14.705407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:14.705407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:14.705407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:14.705407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:14.705407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:14.705407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:14.705407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:14.705407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:14.705407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:14.705407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:14.705407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:14.705407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:14.705407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:14.707174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:14.707174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:14.707174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:14.707174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:14.707174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:14.707174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:14.707174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:14.707174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:14.707174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:14.707174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:14.707174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:14.707174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:14.707174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:14.707174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:14.707174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:14.707174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:14.707174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:14.709972 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:14.709972 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:14.709972 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:14.709972 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 307/605] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:14.713494 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5139d-537d-4a61-be71-c23c955d835c None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:14.847434 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:14.860534 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5139d-537d-4a61-be71-c23c955d835c None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:14.907866 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe00ddf-7360-4ca6-bb78-b927e34d724a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:14.908408 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe00ddf-7360-4ca6-bb78-b927e34d724a None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:14.908679 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe00ddf-7360-4ca6-bb78-b927e34d724a None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:14.908946 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe00ddf-7360-4ca6-bb78-b927e34d724a None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:14.909689 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe00ddf-7360-4ca6-bb78-b927e34d724a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:14.909986 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe00ddf-7360-4ca6-bb78-b927e34d724a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:14.911412 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe00ddf-7360-4ca6-bb78-b927e34d724a None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:14.927618 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 299/606] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:14.933547 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c37c4-538a-4f68-a698-15c5ae719905 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:14.983828 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe00ddf-7360-4ca6-bb78-b927e34d724a None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:15.120668 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:15.132340 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe00ddf-7360-4ca6-bb78-b927e34d724a None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:15.227368 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c37c4-538a-4f68-a698-15c5ae719905 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:15.239201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2d892-5ca4-4442-b413-46b3688cbad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:15.240352 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2d892-5ca4-4442-b413-46b3688cbad0 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:15.245530 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2d892-5ca4-4442-b413-46b3688cbad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:15.245974 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2d892-5ca4-4442-b413-46b3688cbad0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:15.270829 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee2d892-5ca4-4442-b413-46b3688cbad0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:59:15.276170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-0ee2d892-5ca4-4442-b413-46b3688cbad0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:15.276170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:15.276170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:15.276170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:15.276170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:15.276170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:15.276170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:15.276170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:15.276170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:15.276170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:15.276170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:15.276170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:15.276170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:59:15.276170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:59:15.276170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:59:15.276170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:59:15.277224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:59:15.277224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:59:15.277224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:59:15.277224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:59:15.277224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:15.277224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:15.277224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:15.277224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:15.277224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:15.277224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:15.277224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:15.277224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:15.277224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:15.277224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:15.277224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:15.277224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:15.278055 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:15.278055 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:15.278055 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:15.278055 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:15.278055 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:59:15.278055 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:59:15.278055 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:59:15.278055 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:59:15.278055 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:59:15.278055 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:59:15.278055 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:15.278055 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:15.285876 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 300/607] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 14:59:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:15.294248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2d892-5ca4-4442-b413-46b3688cbad0 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:15.316281 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242aa20-4de2-4daa-a6b3-d4bea9ba9ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:15.317416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242aa20-4de2-4daa-a6b3-d4bea9ba9ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:15.317887 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242aa20-4de2-4daa-a6b3-d4bea9ba9ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:15.318353 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242aa20-4de2-4daa-a6b3-d4bea9ba9ca3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:15.326590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:15.346359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242aa20-4de2-4daa-a6b3-d4bea9ba9ca3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:15.355859 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242aa20-4de2-4daa-a6b3-d4bea9ba9ca3 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:15.397345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-6242aa20-4de2-4daa-a6b3-d4bea9ba9ca3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:15.397345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:15.397345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:15.397345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:15.397345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:15.397345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:15.397345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:15.397345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:15.397345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:15.397345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:15.397345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:15.397345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:15.397345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:59:15.397345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:59:15.397345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:59:15.399467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:59:15.399467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:15.399467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:15.399467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:15.399467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:15.399467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:15.399467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:15.399467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:15.399467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:15.399467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:15.399467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:15.399467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:15.399467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:15.399467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:15.399467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:15.399467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:15.399467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:15.401707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:59:15.401707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:59:15.401707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:59:15.401707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:59:15.401707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:59:15.401707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:59:15.401707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:15.401707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:15.401707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 308/608] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:59:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 467 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:15.409168 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242aa20-4de2-4daa-a6b3-d4bea9ba9ca3 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:15.599341 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2d892-5ca4-4442-b413-46b3688cbad0 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:15.653854 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242aa20-4de2-4daa-a6b3-d4bea9ba9ca3 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:15.709093 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1dd31a-4e38-4ca8-8651-6a56c8d0efa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:15.710192 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1dd31a-4e38-4ca8-8651-6a56c8d0efa3 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:15.710673 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1dd31a-4e38-4ca8-8651-6a56c8d0efa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:15.711163 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1dd31a-4e38-4ca8-8651-6a56c8d0efa3 None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:15.712397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1dd31a-4e38-4ca8-8651-6a56c8d0efa3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:15.712927 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1dd31a-4e38-4ca8-8651-6a56c8d0efa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:15.715771 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1dd31a-4e38-4ca8-8651-6a56c8d0efa3 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:15.733920 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:15.738452 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c584ab7-0c47-46cb-97e7-beaed4fe24a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:15.739654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c584ab7-0c47-46cb-97e7-beaed4fe24a7 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:15.740141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c584ab7-0c47-46cb-97e7-beaed4fe24a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:15.740700 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c584ab7-0c47-46cb-97e7-beaed4fe24a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:15.742060 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c584ab7-0c47-46cb-97e7-beaed4fe24a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:15.742603 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c584ab7-0c47-46cb-97e7-beaed4fe24a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:15.745423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c584ab7-0c47-46cb-97e7-beaed4fe24a7 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:15.763812 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 301/609] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 14:59:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 462 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:15.769433 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1dd31a-4e38-4ca8-8651-6a56c8d0efa3 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:15.785270 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 309/610] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 14:59:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:15.790443 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c584ab7-0c47-46cb-97e7-beaed4fe24a7 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:16.013269 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:16.071109 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1dd31a-4e38-4ca8-8651-6a56c8d0efa3 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:16.141509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7754e1-c741-4136-bb6f-2ebb94e6b316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:16.143745 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7754e1-c741-4136-bb6f-2ebb94e6b316 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:16.144820 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7754e1-c741-4136-bb6f-2ebb94e6b316 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:16.148471 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c584ab7-0c47-46cb-97e7-beaed4fe24a7 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:16.150098 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7754e1-c741-4136-bb6f-2ebb94e6b316 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:16.190037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7754e1-c741-4136-bb6f-2ebb94e6b316 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:16.193227 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7754e1-c741-4136-bb6f-2ebb94e6b316 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:16.231795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-6c7754e1-c741-4136-bb6f-2ebb94e6b316 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:16.231795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:16.231795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:16.231795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:16.231795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:16.231795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:16.231795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:16.231795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:16.231795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:16.231795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:16.231795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:16.231795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:16.231795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:16.231795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:16.231795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:16.231795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:16.233648 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:16.233648 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:16.233648 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:16.233648 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:16.233648 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:16.233648 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:16.233648 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:16.233648 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:16.233648 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:16.233648 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:16.233648 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:16.233648 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:16.233648 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:16.233648 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:16.233648 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:16.233648 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:16.233648 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:16.235342 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:16.235342 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:16.235342 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:16.239036 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 302/611] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 471 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:16.255488 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7754e1-c741-4136-bb6f-2ebb94e6b316 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:16.257480 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbca09-3e43-4821-b735-f99db8297e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:16.258240 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbca09-3e43-4821-b735-f99db8297e4d None admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:16.258521 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbca09-3e43-4821-b735-f99db8297e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:16.258822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbca09-3e43-4821-b735-f99db8297e4d None admin] PATH_INFO: `/v3/projects/70ac515938a24c45ae6302dbac46a1ab/users/8b97e2542db54900a99f6f44b78a5e24/roles/ea3b2f1eb283418b8823cb4fff80edba` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:16.283023 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dbca09-3e43-4821-b735-f99db8297e4d None admin] RBAC: Authorizing `identity:create_grant(project_id=70ac515938a24c45ae6302dbac46a1ab, user_id=8b97e2542db54900a99f6f44b78a5e24, role_id=ea3b2f1eb283418b8823cb4fff80edba)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:16.284725 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dbca09-3e43-4821-b735-f99db8297e4d None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:16.306512 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 310/612] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 14:59:15 2022] PUT /identity/v3/projects/70ac515938a24c45ae6302dbac46a1ab/users/8b97e2542db54900a99f6f44b78a5e24/roles/ea3b2f1eb283418b8823cb4fff80edba => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:59:16.451283 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7754e1-c741-4136-bb6f-2ebb94e6b316 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:16.492521 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:16.509482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41c790-6c32-4b06-ae5b-f6a1f6a9d244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:16.510136 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41c790-6c32-4b06-ae5b-f6a1f6a9d244 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:16.510412 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41c790-6c32-4b06-ae5b-f6a1f6a9d244 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:16.510730 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41c790-6c32-4b06-ae5b-f6a1f6a9d244 None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:16.511478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41c790-6c32-4b06-ae5b-f6a1f6a9d244 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:16.511815 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41c790-6c32-4b06-ae5b-f6a1f6a9d244 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:16.513380 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41c790-6c32-4b06-ae5b-f6a1f6a9d244 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:16.534070 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 303/613] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:59:16.540255 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbca09-3e43-4821-b735-f99db8297e4d None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:16.598958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:16.671761 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbca09-3e43-4821-b735-f99db8297e4d None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:16.727769 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe573f2-9350-4100-b36a-c3425fb0572d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:16.728483 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe573f2-9350-4100-b36a-c3425fb0572d None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:16.728768 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe573f2-9350-4100-b36a-c3425fb0572d None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:16.729036 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe573f2-9350-4100-b36a-c3425fb0572d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:16.743989 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe573f2-9350-4100-b36a-c3425fb0572d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:16.745486 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe573f2-9350-4100-b36a-c3425fb0572d None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:16.762499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-7fe573f2-9350-4100-b36a-c3425fb0572d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 14:59:16.762499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:16.762499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:16.762499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:16.762499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:16.762499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:16.762499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:16.762499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:16.762499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:16.762499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:16.762499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:16.762499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:16.762499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:59:16.762499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:59:16.762499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:59:16.762499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:59:16.764267 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:16.764267 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:16.764267 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:59:16.764267 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:16.764267 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:59:16.764267 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:16.764267 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:16.764267 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:16.764267 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:16.764267 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:16.764267 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:16.764267 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:16.764267 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:16.764267 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:16.764267 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:16.764267 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:16.764267 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:16.764267 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:16.766090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:16.766090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:16.766090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:59:16.766090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:59:16.766090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:59:16.766090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:59:16.766090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:59:16.766090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:59:16.766090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 14:59:16.766090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:16.775811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 311/614] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 14:59:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:16.776752 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41c790-6c32-4b06-ae5b-f6a1f6a9d244 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:16.964658 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41c790-6c32-4b06-ae5b-f6a1f6a9d244 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:17.022546 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec3ddc8-dbe1-459a-a7e8-2b215db641c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:17.023558 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3ddc8-dbe1-459a-a7e8-2b215db641c8 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:17.024031 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3ddc8-dbe1-459a-a7e8-2b215db641c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:17.024498 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3ddc8-dbe1-459a-a7e8-2b215db641c8 None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:17.025947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec3ddc8-dbe1-459a-a7e8-2b215db641c8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:17.026470 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec3ddc8-dbe1-459a-a7e8-2b215db641c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:17.033449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec3ddc8-dbe1-459a-a7e8-2b215db641c8 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:17.063245 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 304/615] 10.222.0.44 () {56 vars in 1135 bytes} [Sun Sep 11 14:59:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:59:17.064795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe573f2-9350-4100-b36a-c3425fb0572d None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:17.213317 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe573f2-9350-4100-b36a-c3425fb0572d None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:17.215247 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:17.294255 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2615edf-ba42-484a-ac0d-fd4561eec0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:17.295324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2615edf-ba42-484a-ac0d-fd4561eec0ca None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:17.295798 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2615edf-ba42-484a-ac0d-fd4561eec0ca None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:17.296268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2615edf-ba42-484a-ac0d-fd4561eec0ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:17.325108 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2615edf-ba42-484a-ac0d-fd4561eec0ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:17.327802 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2615edf-ba42-484a-ac0d-fd4561eec0ca None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:17.361650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-e2615edf-ba42-484a-ac0d-fd4561eec0ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:17.361650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:17.361650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:17.361650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:17.361650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:17.361650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:17.361650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:17.361650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:17.361650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:17.361650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:17.361650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:17.361650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:17.361650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:17.361650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:17.361650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:17.361650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:17.364991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:17.364991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:17.364991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:17.364991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:17.364991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:17.364991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:17.364991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:17.364991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:17.364991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:17.364991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:17.364991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:17.364991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:17.364991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:17.364991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:17.364991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:17.364991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:17.364991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:17.368255 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:17.368255 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:17.368255 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:17.368255 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 312/616] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:17.372612 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec3ddc8-dbe1-459a-a7e8-2b215db641c8 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:17.439701 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc665c-5f57-4e63-90f9-894bd9c463c6 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:17.440170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc665c-5f57-4e63-90f9-894bd9c463c6 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:17.440582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc665c-5f57-4e63-90f9-894bd9c463c6 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:17.443028 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 313/617] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:17.455609 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f82598-f5f8-4651-8be1-bc23253939f5 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:17.456026 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f82598-f5f8-4651-8be1-bc23253939f5 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:17.461864 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f82598-f5f8-4651-8be1-bc23253939f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:17.475353 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:17.507269 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-65f82598-f5f8-4651-8be1-bc23253939f5 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:17.515560 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-65f82598-f5f8-4651-8be1-bc23253939f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:17.657509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec3ddc8-dbe1-459a-a7e8-2b215db641c8 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:17.694434 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 314/618] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:59:17.696656 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa834432-d50b-45f7-9b27-99eb30d9f2e1 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:17.697115 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa834432-d50b-45f7-9b27-99eb30d9f2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:17.697624 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa834432-d50b-45f7-9b27-99eb30d9f2e1 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:17.700028 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 315/619] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:17.707741 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70e000-7313-4e1f-91ee-60677b17c006 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:17.708381 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70e000-7313-4e1f-91ee-60677b17c006 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:17.709024 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70e000-7313-4e1f-91ee-60677b17c006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:17.709489 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480a765-8495-4835-b0f7-4a67cd37f1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:17.710127 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480a765-8495-4835-b0f7-4a67cd37f1fe None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:17.710395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480a765-8495-4835-b0f7-4a67cd37f1fe None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:17.710664 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480a765-8495-4835-b0f7-4a67cd37f1fe None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:17.711327 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7480a765-8495-4835-b0f7-4a67cd37f1fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:17.711618 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7480a765-8495-4835-b0f7-4a67cd37f1fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:17.713030 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7480a765-8495-4835-b0f7-4a67cd37f1fe None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:17.727297 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 305/620] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:17.739209 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480a765-8495-4835-b0f7-4a67cd37f1fe None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:17.758653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-8f70e000-7313-4e1f-91ee-60677b17c006 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:17.760521 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-8f70e000-7313-4e1f-91ee-60677b17c006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:17.915857 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 316/621] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:59:17.918389 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7820f-c85d-462d-a0d7-5451b769f9bf None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:17.918835 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7820f-c85d-462d-a0d7-5451b769f9bf None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:17.919283 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7820f-c85d-462d-a0d7-5451b769f9bf None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:17.925191 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 317/622] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:17.929193 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf45a0a-73e9-487b-a843-d91a7a38e701 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:17.930874 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf45a0a-73e9-487b-a843-d91a7a38e701 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:17.931540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf45a0a-73e9-487b-a843-d91a7a38e701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:17.968197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-2bf45a0a-73e9-487b-a843-d91a7a38e701 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:17.969577 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-2bf45a0a-73e9-487b-a843-d91a7a38e701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:17.982648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:17.985645 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480a765-8495-4835-b0f7-4a67cd37f1fe None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:17.989207 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:18.027991 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b5b5d7-bf31-440c-ae00-0f0df897b6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:18.028659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5b5d7-bf31-440c-ae00-0f0df897b6a7 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:18.028938 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5b5d7-bf31-440c-ae00-0f0df897b6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:18.029349 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5b5d7-bf31-440c-ae00-0f0df897b6a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:18.037459 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b5b5d7-bf31-440c-ae00-0f0df897b6a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:18.039193 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b5b5d7-bf31-440c-ae00-0f0df897b6a7 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:18.056198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-38b5b5d7-bf31-440c-ae00-0f0df897b6a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:18.056198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:18.056198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:18.056198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:18.056198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:18.056198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:18.056198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:18.056198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:18.056198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:18.056198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:18.056198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:18.056198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:18.056198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:59:18.056198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:59:18.056198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:59:18.057902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:59:18.057902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:18.057902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:18.057902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:18.057902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:18.057902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:18.057902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:18.057902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:18.057902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:18.057902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:18.057902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:18.057902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:18.057902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:18.057902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:18.057902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:18.057902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:18.057902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:18.060060 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:59:18.060060 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:59:18.060060 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:59:18.060060 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:59:18.060060 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:59:18.060060 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:59:18.060060 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:18.060060 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:18.062496 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 306/623] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:59:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:59:18.063492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b5b5d7-bf31-440c-ae00-0f0df897b6a7 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:18.092174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 318/624] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:59:18.093125 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf45a0a-73e9-487b-a843-d91a7a38e701 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:18.282239 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf45a0a-73e9-487b-a843-d91a7a38e701 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:18.290735 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b5b5d7-bf31-440c-ae00-0f0df897b6a7 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:18.341100 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c76556-08fb-4534-9e08-3a45585cf7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:18.342090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c76556-08fb-4534-9e08-3a45585cf7d7 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:18.342683 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c76556-08fb-4534-9e08-3a45585cf7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:18.342998 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c76556-08fb-4534-9e08-3a45585cf7d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:18.344194 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c76556-08fb-4534-9e08-3a45585cf7d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:18.344696 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c76556-08fb-4534-9e08-3a45585cf7d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:18.347877 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c76556-08fb-4534-9e08-3a45585cf7d7 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:18.355426 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0c3db-01e8-4518-ad89-e3562f5c40dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:18.356068 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0c3db-01e8-4518-ad89-e3562f5c40dc None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:18.356339 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0c3db-01e8-4518-ad89-e3562f5c40dc None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:18.356598 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0c3db-01e8-4518-ad89-e3562f5c40dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:18.368924 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d0c3db-01e8-4518-ad89-e3562f5c40dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:59:18.374338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-71d0c3db-01e8-4518-ad89-e3562f5c40dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:18.374338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:18.374338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:18.374338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:18.374338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:18.374338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:18.374338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:18.374338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:18.374338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:18.374338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:18.374338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:18.374338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:18.374338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:59:18.374338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:59:18.374338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:59:18.374338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:59:18.377710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:59:18.377710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:59:18.377710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:59:18.377710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:59:18.377710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:18.377710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:18.377710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:18.377710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:18.377710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:18.377710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:18.377710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:18.377710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:18.377710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:18.377710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:18.377710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:18.377710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:18.380123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:18.380123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:18.380123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:18.380123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:18.380123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:59:18.380123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:59:18.380123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:59:18.380123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:59:18.380123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:59:18.380123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:59:18.380123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:18.380123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:18.380123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 307/625] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 14:59:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:18.380123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0c3db-01e8-4518-ad89-e3562f5c40dc None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:18.387888 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 319/626] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 14:59:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:18.390530 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c76556-08fb-4534-9e08-3a45585cf7d7 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:18.398981 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:18.406596 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:18.494892 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0c3db-01e8-4518-ad89-e3562f5c40dc None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:18.544559 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55dc3f-df56-4474-bd71-d6270329589b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:18.545540 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55dc3f-df56-4474-bd71-d6270329589b None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:18.545966 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55dc3f-df56-4474-bd71-d6270329589b None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:18.547089 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55dc3f-df56-4474-bd71-d6270329589b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:18.560082 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c76556-08fb-4534-9e08-3a45585cf7d7 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:18.566476 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e55dc3f-df56-4474-bd71-d6270329589b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:59:18.577037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-7e55dc3f-df56-4474-bd71-d6270329589b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:18.577037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:18.577037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:18.577037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:18.577037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:18.577037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:18.577037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:18.577037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:18.577037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:18.577037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:18.577037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:18.577037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:18.577037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:59:18.577037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:59:18.577037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:59:18.577037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:59:18.578210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:59:18.578210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:59:18.578210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:59:18.578210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:59:18.578210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:18.578210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:18.578210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:18.578210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:18.578210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:18.578210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:18.578210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:18.578210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:18.578210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:18.578210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:18.578210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:18.578210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:18.579025 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:18.579025 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:18.579025 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:18.579025 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:18.579025 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:59:18.579025 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:59:18.579025 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:59:18.579025 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:59:18.579025 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:59:18.579025 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:59:18.579025 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:18.579025 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:18.579025 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 308/627] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 14:59:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:18.580332 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55dc3f-df56-4474-bd71-d6270329589b None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:18.603554 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139584b9-404a-43c3-9b5c-e26646a609d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:18.604203 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139584b9-404a-43c3-9b5c-e26646a609d9 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:18.604470 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139584b9-404a-43c3-9b5c-e26646a609d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:18.604741 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139584b9-404a-43c3-9b5c-e26646a609d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:18.605535 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139584b9-404a-43c3-9b5c-e26646a609d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:18.605834 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139584b9-404a-43c3-9b5c-e26646a609d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:18.607264 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139584b9-404a-43c3-9b5c-e26646a609d9 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:18.621182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 320/628] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 14:59:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:18.621182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139584b9-404a-43c3-9b5c-e26646a609d9 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:18.700677 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:18.712236 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:18.736753 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55dc3f-df56-4474-bd71-d6270329589b None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:18.747024 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139584b9-404a-43c3-9b5c-e26646a609d9 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:18.775800 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52339ec2-aa61-4709-b182-cdfdb2ef0cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:18.776469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52339ec2-aa61-4709-b182-cdfdb2ef0cb2 None admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:18.776746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52339ec2-aa61-4709-b182-cdfdb2ef0cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:18.777094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52339ec2-aa61-4709-b182-cdfdb2ef0cb2 None admin] PATH_INFO: `/v3/projects/70ac515938a24c45ae6302dbac46a1ab/users/05aa5e40ee4542719657bc03b2015e72/roles/ea3b2f1eb283418b8823cb4fff80edba` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:18.794501 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52339ec2-aa61-4709-b182-cdfdb2ef0cb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=70ac515938a24c45ae6302dbac46a1ab, user_id=05aa5e40ee4542719657bc03b2015e72, role_id=ea3b2f1eb283418b8823cb4fff80edba)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:18.796051 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52339ec2-aa61-4709-b182-cdfdb2ef0cb2 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:18.798600 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b15c0-5716-44b5-b2fb-bd26d3d3de86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:18.799971 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b15c0-5716-44b5-b2fb-bd26d3d3de86 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:18.800442 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b15c0-5716-44b5-b2fb-bd26d3d3de86 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:18.800781 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b15c0-5716-44b5-b2fb-bd26d3d3de86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:18.821176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 309/629] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 14:59:18 2022] PUT /identity/v3/projects/70ac515938a24c45ae6302dbac46a1ab/users/05aa5e40ee4542719657bc03b2015e72/roles/ea3b2f1eb283418b8823cb4fff80edba => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:59:18.821176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52339ec2-aa61-4709-b182-cdfdb2ef0cb2 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:18.823429 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b15c0-5716-44b5-b2fb-bd26d3d3de86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:59:18.829553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-753b15c0-5716-44b5-b2fb-bd26d3d3de86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:18.829553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:18.829553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:18.829553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:18.829553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:18.829553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:18.829553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:18.829553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:18.829553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:18.829553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:18.829553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:18.829553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:18.829553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:59:18.829553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:59:18.829553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:59:18.829553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:59:18.831996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:59:18.831996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:59:18.831996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:59:18.831996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:59:18.831996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:18.831996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:18.831996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:18.831996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:18.831996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:18.831996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:18.831996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:18.831996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:18.831996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:18.831996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:18.831996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:18.831996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:18.834022 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:18.834022 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:18.834022 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:18.834022 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:18.834022 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:59:18.834022 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:59:18.834022 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:59:18.834022 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:59:18.834022 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:59:18.834022 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:59:18.834022 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:18.834022 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:18.834022 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 321/630] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 14:59:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:18.840515 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b15c0-5716-44b5-b2fb-bd26d3d3de86 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:18.989495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b15c0-5716-44b5-b2fb-bd26d3d3de86 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:19.006584 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52339ec2-aa61-4709-b182-cdfdb2ef0cb2 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:19.033053 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced1d6f-5663-4b56-ba44-79dc8d79e2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:19.033905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced1d6f-5663-4b56-ba44-79dc8d79e2c9 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:19.034382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced1d6f-5663-4b56-ba44-79dc8d79e2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:19.034776 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced1d6f-5663-4b56-ba44-79dc8d79e2c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:19.037553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:19.054187 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ced1d6f-5663-4b56-ba44-79dc8d79e2c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:19.056090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ced1d6f-5663-4b56-ba44-79dc8d79e2c9 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:19.073269 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:19.079091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-0ced1d6f-5663-4b56-ba44-79dc8d79e2c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:19.079091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:19.079091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:19.079091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:19.079091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:19.079091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:19.079091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:19.079091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:19.079091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:19.079091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:19.079091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:19.079091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:19.079091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:19.079091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:19.079091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:19.079091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:19.080926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:19.080926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:19.080926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:19.080926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:19.080926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:19.080926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:19.080926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:19.080926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:19.080926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:19.080926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:19.080926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:19.080926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:19.080926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:19.080926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:19.080926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:19.080926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:19.080926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:19.083744 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:19.083744 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:19.083744 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:19.084377 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e4688-7be2-4be7-a416-d91b4e8b75a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:19.084653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 322/631] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:19.084897 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e4688-7be2-4be7-a416-d91b4e8b75a7 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:19.084897 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e4688-7be2-4be7-a416-d91b4e8b75a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:19.085360 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced1d6f-5663-4b56-ba44-79dc8d79e2c9 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:19.085575 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e4688-7be2-4be7-a416-d91b4e8b75a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:19.085575 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e4688-7be2-4be7-a416-d91b4e8b75a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:19.085575 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e4688-7be2-4be7-a416-d91b4e8b75a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:19.085575 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e4688-7be2-4be7-a416-d91b4e8b75a7 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:19.101006 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 310/632] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 14:59:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:19.101942 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e4688-7be2-4be7-a416-d91b4e8b75a7 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:19.207374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced1d6f-5663-4b56-ba44-79dc8d79e2c9 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:19.281192 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e4688-7be2-4be7-a416-d91b4e8b75a7 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:19.288494 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9829d6-69f9-4646-b803-d3da39f5c2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:19.289598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9829d6-69f9-4646-b803-d3da39f5c2b5 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:19.290077 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9829d6-69f9-4646-b803-d3da39f5c2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:19.290551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9829d6-69f9-4646-b803-d3da39f5c2b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:19.291752 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9829d6-69f9-4646-b803-d3da39f5c2b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:19.292279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9829d6-69f9-4646-b803-d3da39f5c2b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:19.294571 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9829d6-69f9-4646-b803-d3da39f5c2b5 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:19.316864 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 323/633] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 14:59:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:19.319257 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9829d6-69f9-4646-b803-d3da39f5c2b5 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:19.345685 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc15c88-5466-4bb6-a4af-fc371de8d644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:19.346312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc15c88-5466-4bb6-a4af-fc371de8d644 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:19.346963 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc15c88-5466-4bb6-a4af-fc371de8d644 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:19.347229 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc15c88-5466-4bb6-a4af-fc371de8d644 None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:19.348424 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc15c88-5466-4bb6-a4af-fc371de8d644 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:19.348728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc15c88-5466-4bb6-a4af-fc371de8d644 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:19.350667 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc15c88-5466-4bb6-a4af-fc371de8d644 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:19.369259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 311/634] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:19.371311 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc15c88-5466-4bb6-a4af-fc371de8d644 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:19.420312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:19.455624 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:19.512351 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9829d6-69f9-4646-b803-d3da39f5c2b5 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:19.551712 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c88c23-265b-4faa-b260-0b1e910025c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:19.552646 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c88c23-265b-4faa-b260-0b1e910025c4 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:19.553111 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c88c23-265b-4faa-b260-0b1e910025c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:19.553558 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c88c23-265b-4faa-b260-0b1e910025c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:19.554079 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc15c88-5466-4bb6-a4af-fc371de8d644 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:19.567141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c88c23-265b-4faa-b260-0b1e910025c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:19.568892 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c88c23-265b-4faa-b260-0b1e910025c4 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:19.585114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-87c88c23-265b-4faa-b260-0b1e910025c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:19.585114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:19.585114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:19.585114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:19.585114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:19.585114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:19.585114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:19.585114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:19.585114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:19.585114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:19.585114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:19.585114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:19.585114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:19.585114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:19.585114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:19.585114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:19.586403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:19.586403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:19.586403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:19.586403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:19.586403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:19.586403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:19.586403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:19.586403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:19.586403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:19.586403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:19.586403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:19.586403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:19.586403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:19.586403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:19.586403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:19.586403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:19.586403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:19.587439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:19.587439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:19.587439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:19.593611 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 324/635] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:19.594894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c88c23-265b-4faa-b260-0b1e910025c4 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:19.631049 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831bc5d-ea69-4a3a-a74c-c3f3ce3331f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:19.632306 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831bc5d-ea69-4a3a-a74c-c3f3ce3331f3 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:19.632909 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831bc5d-ea69-4a3a-a74c-c3f3ce3331f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:19.633676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831bc5d-ea69-4a3a-a74c-c3f3ce3331f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:19.655974 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4831bc5d-ea69-4a3a-a74c-c3f3ce3331f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:19.659357 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4831bc5d-ea69-4a3a-a74c-c3f3ce3331f3 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:19.692921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-4831bc5d-ea69-4a3a-a74c-c3f3ce3331f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:19.692921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:19.692921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:19.692921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:19.692921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:19.692921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:19.692921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:19.692921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:19.692921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:19.692921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:19.692921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:19.692921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:19.692921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:19.692921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:19.692921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:19.692921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:19.694119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:19.694119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:19.694119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:19.694119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:19.694119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:19.694119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:19.694119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:19.694119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:19.694119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:19.694119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:19.694119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:19.694119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:19.694119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:19.694119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:19.694119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:19.694119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:19.694119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:19.695166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:19.695166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:19.695166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:19.701231 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 312/636] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:19.701231 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831bc5d-ea69-4a3a-a74c-c3f3ce3331f3 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:19.715461 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:19.726849 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c88c23-265b-4faa-b260-0b1e910025c4 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:19.768120 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4481c-53ec-4d75-81d2-0e41cded5539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:19.768779 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4481c-53ec-4d75-81d2-0e41cded5539 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:19.769059 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4481c-53ec-4d75-81d2-0e41cded5539 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:19.769368 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4481c-53ec-4d75-81d2-0e41cded5539 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:19.778397 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc4481c-53ec-4d75-81d2-0e41cded5539 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:19.779881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc4481c-53ec-4d75-81d2-0e41cded5539 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:19.792648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-6bc4481c-53ec-4d75-81d2-0e41cded5539 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 11 14:59:19.792648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:19.792648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:19.792648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:19.792648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:19.792648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:19.792648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:19.792648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:19.792648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:19.792648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:19.792648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:19.792648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:19.792648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:59:19.792648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:59:19.792648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:59:19.792648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:59:19.794458 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:19.794458 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:19.794458 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:59:19.794458 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:19.794458 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:59:19.794458 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:19.794458 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:19.794458 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:19.794458 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:19.794458 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:19.794458 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:19.794458 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:19.794458 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:19.794458 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:19.794458 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:19.794458 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:19.794458 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:19.794458 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:19.796320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:19.796320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:19.796320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:59:19.796320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:59:19.796320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:59:19.796320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:59:19.796320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:59:19.796320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:59:19.796320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 11 14:59:19.796320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:19.798186 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 325/637] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:59:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:19.798744 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4481c-53ec-4d75-81d2-0e41cded5539 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:19.924344 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4481c-53ec-4d75-81d2-0e41cded5539 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:19.924618 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831bc5d-ea69-4a3a-a74c-c3f3ce3331f3 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:19.940831 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:19.968733 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7a386-f4d1-40bb-a515-4de728da1afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:19.969416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7a386-f4d1-40bb-a515-4de728da1afb None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:19.969693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7a386-f4d1-40bb-a515-4de728da1afb None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:19.969988 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7a386-f4d1-40bb-a515-4de728da1afb None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:19.970666 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b7a386-f4d1-40bb-a515-4de728da1afb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:19.970970 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b7a386-f4d1-40bb-a515-4de728da1afb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:19.972786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b7a386-f4d1-40bb-a515-4de728da1afb None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:19.988361 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 326/638] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:19.989370 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7a386-f4d1-40bb-a515-4de728da1afb None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:20.002754 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8fb86-6c30-4eb9-87e9-6b1fba76749c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:20.004189 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8fb86-6c30-4eb9-87e9-6b1fba76749c None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:20.005071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8fb86-6c30-4eb9-87e9-6b1fba76749c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:20.005997 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8fb86-6c30-4eb9-87e9-6b1fba76749c None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:20.007426 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f8fb86-6c30-4eb9-87e9-6b1fba76749c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:20.008230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f8fb86-6c30-4eb9-87e9-6b1fba76749c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:20.008936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:20.013065 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f8fb86-6c30-4eb9-87e9-6b1fba76749c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:20.028429 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 313/639] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:20.030322 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8fb86-6c30-4eb9-87e9-6b1fba76749c None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:20.105428 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7a386-f4d1-40bb-a515-4de728da1afb None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:20.158974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5fee9-7f7f-451c-8102-c521a8efc839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:20.159849 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5fee9-7f7f-451c-8102-c521a8efc839 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:20.160289 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5fee9-7f7f-451c-8102-c521a8efc839 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:20.160723 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5fee9-7f7f-451c-8102-c521a8efc839 None admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:20.161921 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5fee9-7f7f-451c-8102-c521a8efc839 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:20.162415 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5fee9-7f7f-451c-8102-c521a8efc839 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:20.164661 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5fee9-7f7f-451c-8102-c521a8efc839 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:20.176040 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8fb86-6c30-4eb9-87e9-6b1fba76749c None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:20.190636 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 327/640] 10.222.0.44 () {56 vars in 1137 bytes} [Sun Sep 11 14:59:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:20.192722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f12751-c97d-4945-abe0-e8eaf358ea19 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:20.193253 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f12751-c97d-4945-abe0-e8eaf358ea19 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:20.193745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f12751-c97d-4945-abe0-e8eaf358ea19 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:20.195814 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 328/641] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:20.197191 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f12751-c97d-4945-abe0-e8eaf358ea19 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:20.226656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154a1dc-47a2-4e8f-89be-5983280d117e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:20.227299 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154a1dc-47a2-4e8f-89be-5983280d117e None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:20.227579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154a1dc-47a2-4e8f-89be-5983280d117e None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:20.227836 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154a1dc-47a2-4e8f-89be-5983280d117e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:20.238288 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4154a1dc-47a2-4e8f-89be-5983280d117e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:20.240049 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4154a1dc-47a2-4e8f-89be-5983280d117e None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:20.260331 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-4154a1dc-47a2-4e8f-89be-5983280d117e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 14:59:20.260331 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:20.260331 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:20.260331 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:20.260331 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:20.260331 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:20.260331 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:20.260331 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:20.260331 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:20.260331 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:20.260331 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:20.260331 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:20.260331 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:59:20.260331 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:59:20.260331 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:59:20.260331 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:59:20.261469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:20.261469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:20.261469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:59:20.261469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:20.261469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:59:20.261469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:20.261469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:20.261469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:20.261469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:20.261469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:20.261469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:20.261469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:20.261469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:20.261469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:20.261469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:20.261469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:20.261469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:20.261469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:20.262402 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:20.262402 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:20.262402 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:59:20.262402 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:59:20.262402 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:59:20.262402 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:59:20.262402 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:59:20.262402 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:59:20.262402 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 14:59:20.262402 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:20.262402 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 314/642] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 14:59:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:20.269062 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154a1dc-47a2-4e8f-89be-5983280d117e None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:20.307201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:20.309628 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:20.345482 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f12751-c97d-4945-abe0-e8eaf358ea19 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:20.387506 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026824f2-14a7-4ff1-a68a-6fcdbbf2b537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:20.388327 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026824f2-14a7-4ff1-a68a-6fcdbbf2b537 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:20.388768 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026824f2-14a7-4ff1-a68a-6fcdbbf2b537 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:20.389190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026824f2-14a7-4ff1-a68a-6fcdbbf2b537 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:20.402356 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026824f2-14a7-4ff1-a68a-6fcdbbf2b537 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:20.404746 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026824f2-14a7-4ff1-a68a-6fcdbbf2b537 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:20.407453 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154a1dc-47a2-4e8f-89be-5983280d117e None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:20.419857 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-026824f2-14a7-4ff1-a68a-6fcdbbf2b537 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:59:20.419857 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:20.419857 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:20.419857 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:20.419857 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:20.419857 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:20.419857 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:20.419857 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:20.419857 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:20.419857 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:20.419857 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:20.419857 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:20.419857 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:59:20.419857 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:59:20.419857 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:59:20.419857 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:59:20.421751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:20.421751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:20.421751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:59:20.421751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:20.421751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:59:20.421751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:20.421751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:20.421751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:20.421751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:20.421751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:20.421751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:20.421751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:20.421751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:20.421751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:20.421751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:20.421751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:20.421751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:20.421751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:20.423673 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:20.423673 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:20.423673 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:59:20.423673 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:59:20.423673 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:59:20.423673 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:59:20.423673 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:59:20.423673 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:59:20.423673 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:59:20.423673 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:20.429029 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 329/643] 10.222.0.44 () {56 vars in 1093 bytes} [Sun Sep 11 14:59:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:20.430488 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8d90c-ed46-44fc-8f9b-144a9b0d8eea None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:20.430940 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8d90c-ed46-44fc-8f9b-144a9b0d8eea None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:20.431343 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8d90c-ed46-44fc-8f9b-144a9b0d8eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:20.449820 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa41bc8-a14e-4cff-9833-96bc388fa65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:20.450434 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa41bc8-a14e-4cff-9833-96bc388fa65b None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:20.450704 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa41bc8-a14e-4cff-9833-96bc388fa65b None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:20.450966 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa41bc8-a14e-4cff-9833-96bc388fa65b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:20.453265 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-8da8d90c-ed46-44fc-8f9b-144a9b0d8eea None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:20.454805 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-8da8d90c-ed46-44fc-8f9b-144a9b0d8eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:20.530131 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa41bc8-a14e-4cff-9833-96bc388fa65b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:20.531835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa41bc8-a14e-4cff-9833-96bc388fa65b None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:20.545644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 330/644] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:59:20.546911 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8d90c-ed46-44fc-8f9b-144a9b0d8eea None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:20.600958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:20.619090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:20.693184 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8d90c-ed46-44fc-8f9b-144a9b0d8eea None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:20.753547 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e9aab-8f72-473d-8d2a-12f6f1ba9240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:20.754550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e9aab-8f72-473d-8d2a-12f6f1ba9240 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:20.755006 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e9aab-8f72-473d-8d2a-12f6f1ba9240 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:20.755460 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e9aab-8f72-473d-8d2a-12f6f1ba9240 None admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:20.756691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e9aab-8f72-473d-8d2a-12f6f1ba9240 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:20.757242 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e9aab-8f72-473d-8d2a-12f6f1ba9240 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:20.759795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e9aab-8f72-473d-8d2a-12f6f1ba9240 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:20.783023 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 315/645] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:59:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:59:20.783023 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa41bc8-a14e-4cff-9833-96bc388fa65b None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:20.803369 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 331/646] 10.222.0.44 () {56 vars in 1139 bytes} [Sun Sep 11 14:59:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:20.805225 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e9aab-8f72-473d-8d2a-12f6f1ba9240 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:20.962768 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa41bc8-a14e-4cff-9833-96bc388fa65b None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:20.985481 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e9aab-8f72-473d-8d2a-12f6f1ba9240 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:21.007898 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:21.017849 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:21.022558 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87780b-925d-427f-93cf-26e71618019a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:21.023565 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87780b-925d-427f-93cf-26e71618019a None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:21.023968 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87780b-925d-427f-93cf-26e71618019a None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:21.024360 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87780b-925d-427f-93cf-26e71618019a None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:21.025518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce87780b-925d-427f-93cf-26e71618019a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:21.026270 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce87780b-925d-427f-93cf-26e71618019a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:21.028188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce87780b-925d-427f-93cf-26e71618019a None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:21.040164 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e893f-d06c-47a9-9236-2f6b2b17983d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:21.042181 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e893f-d06c-47a9-9236-2f6b2b17983d None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:21.042950 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e893f-d06c-47a9-9236-2f6b2b17983d None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:21.043673 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e893f-d06c-47a9-9236-2f6b2b17983d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:21.059954 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 316/647] 10.222.0.44 () {56 vars in 1133 bytes} [Sun Sep 11 14:59:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:21.059954 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87780b-925d-427f-93cf-26e71618019a None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:21.068086 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e893f-d06c-47a9-9236-2f6b2b17983d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:21.069877 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e893f-d06c-47a9-9236-2f6b2b17983d None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:21.090123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-910e893f-d06c-47a9-9236-2f6b2b17983d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:21.090123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:21.090123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:21.090123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:21.090123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:21.090123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:21.090123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:21.090123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:21.090123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:21.090123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:21.090123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:21.090123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:21.090123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:21.090123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:21.090123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:21.090123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:21.091932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:21.091932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:21.091932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:21.091932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:21.091932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:21.091932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:21.091932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:21.091932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:21.091932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:21.091932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:21.091932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:21.091932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:21.091932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:21.091932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:21.091932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:21.091932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:21.091932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:21.094130 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:21.094130 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:21.094130 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:21.094130 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 332/648] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:21.094130 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e893f-d06c-47a9-9236-2f6b2b17983d None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:21.186835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87780b-925d-427f-93cf-26e71618019a None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:21.217823 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e893f-d06c-47a9-9236-2f6b2b17983d None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:21.223799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e29988-5a29-43ac-8fb1-762ce42e9584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:21.224426 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e29988-5a29-43ac-8fb1-762ce42e9584 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:21.224704 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e29988-5a29-43ac-8fb1-762ce42e9584 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:21.224977 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e29988-5a29-43ac-8fb1-762ce42e9584 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:21.258757 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d405d-34a8-455a-875f-c16ff15fc55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:21.259411 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d405d-34a8-455a-875f-c16ff15fc55b None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:21.259693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d405d-34a8-455a-875f-c16ff15fc55b None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:21.259957 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d405d-34a8-455a-875f-c16ff15fc55b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:21.273242 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d405d-34a8-455a-875f-c16ff15fc55b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:59:21.278739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-8f2d405d-34a8-455a-875f-c16ff15fc55b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:21.278739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:21.278739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:21.278739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:21.278739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:21.278739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:21.278739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:21.278739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:21.278739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:21.278739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:21.278739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:21.278739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:21.278739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:59:21.278739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:59:21.278739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:59:21.278739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:59:21.279736 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:59:21.279736 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:59:21.279736 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:59:21.279736 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:59:21.279736 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:21.279736 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:21.279736 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:21.279736 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:21.279736 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:21.279736 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:21.279736 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:21.279736 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:21.279736 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:21.279736 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:21.279736 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:21.279736 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:21.280578 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:21.280578 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:21.280578 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:21.280578 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:21.280578 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:59:21.280578 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:59:21.280578 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:59:21.280578 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:59:21.280578 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:59:21.280578 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:59:21.280578 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:21.280578 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:21.280578 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 333/649] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 14:59:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:21.281284 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d405d-34a8-455a-875f-c16ff15fc55b None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:21.294664 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:21.314007 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e29988-5a29-43ac-8fb1-762ce42e9584 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:21.315536 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e29988-5a29-43ac-8fb1-762ce42e9584 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:21.318899 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:21.395773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d405d-34a8-455a-875f-c16ff15fc55b None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:21.428300 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 317/650] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:59:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:59:21.428300 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e29988-5a29-43ac-8fb1-762ce42e9584 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:21.433495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24139c-5626-4762-9df6-4ab36bf1168e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:21.433586 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24139c-5626-4762-9df6-4ab36bf1168e None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:21.433868 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24139c-5626-4762-9df6-4ab36bf1168e None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:21.434134 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24139c-5626-4762-9df6-4ab36bf1168e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:21.506309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf24139c-5626-4762-9df6-4ab36bf1168e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:21.507674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf24139c-5626-4762-9df6-4ab36bf1168e None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:21.523396 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:21.542969 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:21.543238 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e29988-5a29-43ac-8fb1-762ce42e9584 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:21.580430 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84a68f-001e-4577-ae53-57e7ed4e7956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:21.581044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84a68f-001e-4577-ae53-57e7ed4e7956 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:21.581351 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84a68f-001e-4577-ae53-57e7ed4e7956 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:21.581618 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84a68f-001e-4577-ae53-57e7ed4e7956 None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:21.582282 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c84a68f-001e-4577-ae53-57e7ed4e7956 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:21.582573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c84a68f-001e-4577-ae53-57e7ed4e7956 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:21.583973 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c84a68f-001e-4577-ae53-57e7ed4e7956 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:21.596122 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 318/651] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:21.597894 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420edb3-e01e-42e5-acdc-17a10df82aa0 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:21.598376 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420edb3-e01e-42e5-acdc-17a10df82aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:21.598565 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420edb3-e01e-42e5-acdc-17a10df82aa0 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:21.599715 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 319/652] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:21.600447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420edb3-e01e-42e5-acdc-17a10df82aa0 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:21.623804 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 334/653] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:59:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:59:21.625540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24139c-5626-4762-9df6-4ab36bf1168e None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:21.712514 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420edb3-e01e-42e5-acdc-17a10df82aa0 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:21.737848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24139c-5626-4762-9df6-4ab36bf1168e None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:21.750340 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa31d6e-3b24-4174-89bb-e966db7d650b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:21.751114 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa31d6e-3b24-4174-89bb-e966db7d650b None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:21.751470 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa31d6e-3b24-4174-89bb-e966db7d650b None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:21.751874 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa31d6e-3b24-4174-89bb-e966db7d650b None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:21.752631 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa31d6e-3b24-4174-89bb-e966db7d650b None admin] RBAC: Adding query filter params (name=service) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:21.753007 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa31d6e-3b24-4174-89bb-e966db7d650b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:21.754680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa31d6e-3b24-4174-89bb-e966db7d650b None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:21.763703 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 320/654] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 14:59:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:21.764557 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa31d6e-3b24-4174-89bb-e966db7d650b None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:21.770321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:21.773806 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:21.778303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3223cf-88aa-41b6-af3e-ede4defc343b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:21.779112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3223cf-88aa-41b6-af3e-ede4defc343b None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:21.779535 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3223cf-88aa-41b6-af3e-ede4defc343b None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:21.779872 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3223cf-88aa-41b6-af3e-ede4defc343b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:21.792218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3223cf-88aa-41b6-af3e-ede4defc343b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:21.794204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3223cf-88aa-41b6-af3e-ede4defc343b None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:21.818204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-6f3223cf-88aa-41b6-af3e-ede4defc343b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:21.818204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:21.818204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:21.818204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:21.818204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:21.818204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:21.818204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:21.818204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:21.818204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:21.818204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:21.818204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:21.818204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:21.818204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:21.818204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:21.818204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:21.818204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:21.819332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:21.819332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:21.819332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:21.819332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:21.819332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:21.819332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:21.819332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:21.819332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:21.819332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:21.819332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:21.819332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:21.819332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:21.819332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:21.819332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:21.819332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:21.819332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:21.819332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:21.820225 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:21.820225 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:21.820225 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:21.820225 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 335/655] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:21.821570 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff112b86-22de-47b7-a9ff-8d9de7192db2 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:21.821884 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff112b86-22de-47b7-a9ff-8d9de7192db2 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:21.822191 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff112b86-22de-47b7-a9ff-8d9de7192db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:21.840757 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-ff112b86-22de-47b7-a9ff-8d9de7192db2 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:21.842013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-ff112b86-22de-47b7-a9ff-8d9de7192db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:21.889014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa31d6e-3b24-4174-89bb-e966db7d650b None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:21.937162 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 336/656] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:59:21.937162 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff112b86-22de-47b7-a9ff-8d9de7192db2 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:21.937475 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3a7a3-7abe-4e38-8272-b437fbe3f294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:21.937475 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3a7a3-7abe-4e38-8272-b437fbe3f294 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:21.937475 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3a7a3-7abe-4e38-8272-b437fbe3f294 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:21.937475 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3a7a3-7abe-4e38-8272-b437fbe3f294 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:21.937751 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3a7a3-7abe-4e38-8272-b437fbe3f294 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:21.937751 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3a7a3-7abe-4e38-8272-b437fbe3f294 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:21.956755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-4cc3a7a3-7abe-4e38-8272-b437fbe3f294 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:21.956755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:21.956755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:21.956755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:21.956755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:21.956755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:21.956755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:21.956755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:21.956755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:21.956755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:21.956755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:21.956755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:21.956755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:59:21.956755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:59:21.956755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:59:21.958508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:59:21.958508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:21.958508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:21.958508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:21.958508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:21.958508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:21.958508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:21.958508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:21.958508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:21.958508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:21.958508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:21.958508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:21.958508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:21.958508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:21.958508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:21.958508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:21.958508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:21.961537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:59:21.961537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:59:21.961537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:59:21.961537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:59:21.961537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:59:21.961537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:59:21.961537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:21.961537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:21.961537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 321/657] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:59:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:21.961537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3a7a3-7abe-4e38-8272-b437fbe3f294 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:22.044950 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:22.060637 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff112b86-22de-47b7-a9ff-8d9de7192db2 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:22.077916 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:22.092693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3a7a3-7abe-4e38-8272-b437fbe3f294 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:22.105586 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac68ab-e0d1-4640-9b19-3950bb7697a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:22.106373 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac68ab-e0d1-4640-9b19-3950bb7697a8 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:22.106768 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac68ab-e0d1-4640-9b19-3950bb7697a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:22.107131 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac68ab-e0d1-4640-9b19-3950bb7697a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:22.120797 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac68ab-e0d1-4640-9b19-3950bb7697a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:22.122406 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac68ab-e0d1-4640-9b19-3950bb7697a8 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:22.135466 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0466d-5ad2-402f-9238-95227aae98e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:22.136240 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0466d-5ad2-402f-9238-95227aae98e1 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:22.136688 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0466d-5ad2-402f-9238-95227aae98e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:22.137007 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0466d-5ad2-402f-9238-95227aae98e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:22.139945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-5fac68ab-e0d1-4640-9b19-3950bb7697a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:22.139945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:22.139945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:22.139945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:22.139945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:22.139945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:22.139945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:22.139945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:22.139945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:22.139945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:22.139945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:22.139945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:22.139945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:22.139945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:22.139945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:22.139945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:22.141928 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:22.141928 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:22.141928 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:22.141928 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:22.141928 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:22.141928 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:22.141928 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:22.141928 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:22.141928 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:22.141928 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:22.141928 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:22.141928 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:22.141928 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:22.141928 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:22.141928 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:22.141928 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:22.141928 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:22.143861 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:22.143861 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:22.143861 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:22.143861 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 337/658] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:22.147284 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac68ab-e0d1-4640-9b19-3950bb7697a8 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:22.154451 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0466d-5ad2-402f-9238-95227aae98e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:22.156071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0466d-5ad2-402f-9238-95227aae98e1 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:22.172154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-b4a0466d-5ad2-402f-9238-95227aae98e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:22.172154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:22.172154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:22.172154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:22.172154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:22.172154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:22.172154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:22.172154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:22.172154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:22.172154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:22.172154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:22.172154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:22.172154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:22.172154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:22.172154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:22.172154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:22.174091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:22.174091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:22.174091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:22.174091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:22.174091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:22.174091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:22.174091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:22.174091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:22.174091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:22.174091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:22.174091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:22.174091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:22.174091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:22.174091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:22.174091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:22.174091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:22.174091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:22.175933 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:22.175933 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:22.175933 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:22.178796 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 322/659] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:22.179661 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0466d-5ad2-402f-9238-95227aae98e1 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:22.274655 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac68ab-e0d1-4640-9b19-3950bb7697a8 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:22.307633 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0466d-5ad2-402f-9238-95227aae98e1 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:22.317059 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002cd1f0-4abf-4050-9b19-1278f41faf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:22.317683 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002cd1f0-4abf-4050-9b19-1278f41faf1d None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:22.317974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002cd1f0-4abf-4050-9b19-1278f41faf1d None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:22.318236 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002cd1f0-4abf-4050-9b19-1278f41faf1d None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:22.318885 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002cd1f0-4abf-4050-9b19-1278f41faf1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:22.319216 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002cd1f0-4abf-4050-9b19-1278f41faf1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:22.320641 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002cd1f0-4abf-4050-9b19-1278f41faf1d None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:22.327798 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:22.334853 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 338/660] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:22.336293 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002cd1f0-4abf-4050-9b19-1278f41faf1d None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:22.357692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647e09a-db0f-4fab-958b-795efa138609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:22.358296 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647e09a-db0f-4fab-958b-795efa138609 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:22.358579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647e09a-db0f-4fab-958b-795efa138609 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:22.358841 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647e09a-db0f-4fab-958b-795efa138609 None admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:22.359554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4647e09a-db0f-4fab-958b-795efa138609 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:22.359849 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4647e09a-db0f-4fab-958b-795efa138609 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:22.361323 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4647e09a-db0f-4fab-958b-795efa138609 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:22.368669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:22.374954 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 323/661] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 14:59:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:22.380099 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647e09a-db0f-4fab-958b-795efa138609 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:22.465567 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002cd1f0-4abf-4050-9b19-1278f41faf1d None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:22.500326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647e09a-db0f-4fab-958b-795efa138609 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:22.512774 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf98496-4def-4767-bb49-58a0c6bc59cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:22.513507 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf98496-4def-4767-bb49-58a0c6bc59cf None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:22.513835 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf98496-4def-4767-bb49-58a0c6bc59cf None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:22.514180 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf98496-4def-4767-bb49-58a0c6bc59cf None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:22.514954 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf98496-4def-4767-bb49-58a0c6bc59cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:22.515326 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf98496-4def-4767-bb49-58a0c6bc59cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:22.516871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf98496-4def-4767-bb49-58a0c6bc59cf None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:22.529385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 339/662] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:22.530244 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf98496-4def-4767-bb49-58a0c6bc59cf None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:22.539339 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69b69c-516b-4f35-92a4-53be4b344dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:22.540045 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69b69c-516b-4f35-92a4-53be4b344dde None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:22.540365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69b69c-516b-4f35-92a4-53be4b344dde None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:22.541071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69b69c-516b-4f35-92a4-53be4b344dde None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:22.541540 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c69b69c-516b-4f35-92a4-53be4b344dde None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:22.541950 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c69b69c-516b-4f35-92a4-53be4b344dde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:22.543571 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c69b69c-516b-4f35-92a4-53be4b344dde None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:22.558749 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 324/663] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:22.559879 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69b69c-516b-4f35-92a4-53be4b344dde None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:22.607652 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:22.625356 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:22.644219 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf98496-4def-4767-bb49-58a0c6bc59cf None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:22.677366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69b69c-516b-4f35-92a4-53be4b344dde None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:22.686994 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a019c3-1198-4d04-aa1e-0d0e291307d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:22.687965 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a019c3-1198-4d04-aa1e-0d0e291307d3 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:22.688405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a019c3-1198-4d04-aa1e-0d0e291307d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:22.688835 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a019c3-1198-4d04-aa1e-0d0e291307d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:22.705819 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a019c3-1198-4d04-aa1e-0d0e291307d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:59:22.714905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-69a019c3-1198-4d04-aa1e-0d0e291307d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:22.714905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:22.714905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:22.714905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:22.714905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:22.714905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:22.714905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:22.714905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:22.714905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:22.714905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:22.714905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:22.714905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:22.714905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:59:22.714905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:59:22.714905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:59:22.714905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:59:22.716174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:59:22.716174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:59:22.716174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:59:22.716174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:59:22.716174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:22.716174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:22.716174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:22.716174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:22.716174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:22.716174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:22.716174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:22.716174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:22.716174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:22.716174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:22.716174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:22.716174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:22.717009 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:22.717009 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:22.717009 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:22.717009 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:22.717009 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:59:22.717009 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:59:22.717009 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:59:22.717009 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:59:22.717009 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:59:22.717009 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:59:22.717009 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:22.717009 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:22.717009 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 340/664] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 14:59:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:22.717855 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a019c3-1198-4d04-aa1e-0d0e291307d3 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:22.731306 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6ffac-9db6-46dd-b4c1-dc268ddb6195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:22.732292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6ffac-9db6-46dd-b4c1-dc268ddb6195 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:22.732732 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6ffac-9db6-46dd-b4c1-dc268ddb6195 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:22.733169 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6ffac-9db6-46dd-b4c1-dc268ddb6195 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:22.744133 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6ffac-9db6-46dd-b4c1-dc268ddb6195 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:22.746780 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6ffac-9db6-46dd-b4c1-dc268ddb6195 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:22.766312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-33a6ffac-9db6-46dd-b4c1-dc268ddb6195 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:22.766312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:22.766312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:22.766312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:22.766312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:22.766312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:22.766312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:22.766312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:22.766312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:22.766312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:22.766312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:22.766312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:22.766312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:59:22.766312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:59:22.766312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:59:22.767303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:59:22.767303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:22.767303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:22.767303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:22.767303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:22.767303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:22.767303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:22.767303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:22.767303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:22.767303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:22.767303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:22.767303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:22.767303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:22.767303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:22.767303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:22.767303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:22.767303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:22.768178 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:59:22.768178 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:59:22.768178 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:59:22.768178 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:59:22.768178 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:59:22.768178 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:59:22.768178 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:22.768178 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:22.768178 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 325/665] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:59:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:22.769594 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6ffac-9db6-46dd-b4c1-dc268ddb6195 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:22.868079 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a019c3-1198-4d04-aa1e-0d0e291307d3 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:22.882883 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:22.887860 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6ffac-9db6-46dd-b4c1-dc268ddb6195 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:22.915927 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe9ad4-f18d-419a-b823-368906829745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:22.916784 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe9ad4-f18d-419a-b823-368906829745 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:22.917173 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe9ad4-f18d-419a-b823-368906829745 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:22.917960 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe9ad4-f18d-419a-b823-368906829745 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:22.918842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe9ad4-f18d-419a-b823-368906829745 None admin] RBAC: Adding query filter params (role.id=ea3b2f1eb283418b8823cb4fff80edba, scope.project.id=70ac515938a24c45ae6302dbac46a1ab, user.id=a1fe5293447346eaafd16818e5bd12f4) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:22.919297 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe9ad4-f18d-419a-b823-368906829745 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:22.921164 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe9ad4-f18d-419a-b823-368906829745 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:22.925887 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c0026-0276-42bf-b41a-d36086380294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:22.926537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c0026-0276-42bf-b41a-d36086380294 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:22.926839 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c0026-0276-42bf-b41a-d36086380294 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:22.927108 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c0026-0276-42bf-b41a-d36086380294 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:22.941192 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 341/666] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:59:22 2022] GET /identity/v3/role_assignments?user.id=a1fe5293447346eaafd16818e5bd12f4&scope.project.id=70ac515938a24c45ae6302dbac46a1ab&role.id=ea3b2f1eb283418b8823cb4fff80edba => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:22.941192 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe9ad4-f18d-419a-b823-368906829745 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:22.941582 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c0026-0276-42bf-b41a-d36086380294 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:22.941582 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c0026-0276-42bf-b41a-d36086380294 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:22.949482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-0d3c0026-0276-42bf-b41a-d36086380294 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:22.949482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:22.949482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:22.949482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:22.949482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:22.949482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:22.949482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:22.949482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:22.949482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:22.949482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:22.949482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:22.949482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:22.949482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:59:22.949482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:59:22.949482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:59:22.951160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:59:22.951160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:22.951160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:22.951160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:22.951160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:22.951160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:22.951160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:22.951160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:22.951160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:22.951160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:22.951160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:22.951160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:22.951160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:22.951160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:22.951160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:22.951160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:22.951160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:22.952822 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:59:22.952822 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:59:22.952822 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:59:22.952822 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:59:22.952822 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:59:22.952822 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:59:22.952822 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:22.952822 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:22.952822 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 326/667] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:59:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:22.952822 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c0026-0276-42bf-b41a-d36086380294 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:22.955245 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:23.132971 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe9ad4-f18d-419a-b823-368906829745 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:23.142565 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c0026-0276-42bf-b41a-d36086380294 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:23.189571 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516e105-a3d2-4937-8523-d91a4290acc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:23.190227 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516e105-a3d2-4937-8523-d91a4290acc9 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:23.190506 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516e105-a3d2-4937-8523-d91a4290acc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:23.190778 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516e105-a3d2-4937-8523-d91a4290acc9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:23.192811 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd9dad-1235-4841-8bda-9dcccc918b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:23.193556 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd9dad-1235-4841-8bda-9dcccc918b99 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:23.193979 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd9dad-1235-4841-8bda-9dcccc918b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:23.194306 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd9dad-1235-4841-8bda-9dcccc918b99 None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:23.195045 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd9dad-1235-4841-8bda-9dcccc918b99 None admin] RBAC: Adding query filter params (name=service) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:23.195420 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd9dad-1235-4841-8bda-9dcccc918b99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:23.196894 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efd9dad-1235-4841-8bda-9dcccc918b99 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:23.200808 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516e105-a3d2-4937-8523-d91a4290acc9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:23.202320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516e105-a3d2-4937-8523-d91a4290acc9 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:23.209184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 327/668] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 14:59:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:23.210952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd9dad-1235-4841-8bda-9dcccc918b99 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:23.217392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-1516e105-a3d2-4937-8523-d91a4290acc9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 14:59:23.217392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:23.217392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:23.217392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:23.217392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:23.217392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:23.217392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:23.217392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:23.217392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:23.217392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:23.217392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:23.217392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:23.217392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:59:23.217392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:59:23.217392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:59:23.217392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:59:23.219274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:23.219274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:23.219274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:59:23.219274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:23.219274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:59:23.219274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:23.219274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:23.219274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:23.219274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:23.219274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:23.219274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:23.219274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:23.219274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:23.219274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:23.219274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:23.219274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:23.219274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:23.219274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:23.221690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:23.221690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:23.221690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:59:23.221690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:59:23.221690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:59:23.221690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:59:23.221690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:59:23.221690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:59:23.221690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 14:59:23.221690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:23.221690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 342/669] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:59:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:23.221690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516e105-a3d2-4937-8523-d91a4290acc9 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:23.250210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:23.345610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:23.382260 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd9dad-1235-4841-8bda-9dcccc918b99 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:23.411171 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516e105-a3d2-4937-8523-d91a4290acc9 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:23.442048 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d5390-cd58-40ed-85b7-7635044c94a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:23.442953 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d5390-cd58-40ed-85b7-7635044c94a6 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:23.443367 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d5390-cd58-40ed-85b7-7635044c94a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:23.443793 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d5390-cd58-40ed-85b7-7635044c94a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:23.444811 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d5390-cd58-40ed-85b7-7635044c94a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:23.445346 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d5390-cd58-40ed-85b7-7635044c94a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:23.447837 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d5390-cd58-40ed-85b7-7635044c94a6 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:23.475143 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 328/670] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 14:59:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:23.475143 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d5390-cd58-40ed-85b7-7635044c94a6 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:23.493262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2a0db-350b-4a63-8789-dd8341697018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:23.494163 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2a0db-350b-4a63-8789-dd8341697018 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:23.494543 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2a0db-350b-4a63-8789-dd8341697018 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:23.494934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2a0db-350b-4a63-8789-dd8341697018 None admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:23.496164 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a2a0db-350b-4a63-8789-dd8341697018 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:23.496640 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a2a0db-350b-4a63-8789-dd8341697018 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:23.498782 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a2a0db-350b-4a63-8789-dd8341697018 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:23.528310 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 343/671] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 14:59:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:23.528310 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2a0db-350b-4a63-8789-dd8341697018 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:23.611397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:23.647482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d5390-cd58-40ed-85b7-7635044c94a6 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:23.682140 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2a0db-350b-4a63-8789-dd8341697018 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:23.691321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b100be-3a62-491f-959d-038cb1b40e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:23.691962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b100be-3a62-491f-959d-038cb1b40e9d None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:23.692241 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b100be-3a62-491f-959d-038cb1b40e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:23.692506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b100be-3a62-491f-959d-038cb1b40e9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:23.712251 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b100be-3a62-491f-959d-038cb1b40e9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:23.715123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b100be-3a62-491f-959d-038cb1b40e9d None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:23.719693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:23.730389 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6fb230-b768-4ebe-b09e-eeff0d25ce05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:23.731229 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fb230-b768-4ebe-b09e-eeff0d25ce05 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:23.732261 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fb230-b768-4ebe-b09e-eeff0d25ce05 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:23.732649 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fb230-b768-4ebe-b09e-eeff0d25ce05 None admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:23.733847 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6fb230-b768-4ebe-b09e-eeff0d25ce05 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:23.734016 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6fb230-b768-4ebe-b09e-eeff0d25ce05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:23.735475 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6fb230-b768-4ebe-b09e-eeff0d25ce05 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:23.743903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-b4b100be-3a62-491f-959d-038cb1b40e9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:23.743903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:23.743903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:23.743903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:23.743903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:23.743903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:23.743903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:23.743903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:23.743903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:23.743903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:23.743903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:23.743903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:23.743903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:23.743903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:23.743903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:23.743903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:23.746071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:23.746071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:23.746071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:23.746071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:23.746071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:23.746071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:23.746071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:23.746071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:23.746071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:23.746071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:23.746071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:23.746071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:23.746071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:23.746071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:23.746071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:23.746071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:23.746071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:23.748720 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:23.748720 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:23.748720 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:23.748720 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 329/672] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:23.748720 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b100be-3a62-491f-959d-038cb1b40e9d None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:23.765285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 344/673] 10.222.0.44 () {56 vars in 1137 bytes} [Sun Sep 11 14:59:23 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:23.766373 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6fb230-b768-4ebe-b09e-eeff0d25ce05 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:23.928443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b100be-3a62-491f-959d-038cb1b40e9d None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:23.942478 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6fb230-b768-4ebe-b09e-eeff0d25ce05 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:23.972538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:23.979015 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff5ef0-a0e0-4535-a190-c09de48a72ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:23.979637 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff5ef0-a0e0-4535-a190-c09de48a72ce None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:23.979919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff5ef0-a0e0-4535-a190-c09de48a72ce None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:23.980233 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff5ef0-a0e0-4535-a190-c09de48a72ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:23.980995 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff5ef0-a0e0-4535-a190-c09de48a72ce None admin] RBAC: Adding query filter params (role.id=ea3b2f1eb283418b8823cb4fff80edba, scope.project.id=70ac515938a24c45ae6302dbac46a1ab, user.id=697b73f182e6419a84e0c8c6b0fc0c06) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:23.981336 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff5ef0-a0e0-4535-a190-c09de48a72ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:23.982864 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff5ef0-a0e0-4535-a190-c09de48a72ce None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:23.990045 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 330/674] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:59:23 2022] GET /identity/v3/role_assignments?user.id=697b73f182e6419a84e0c8c6b0fc0c06&scope.project.id=70ac515938a24c45ae6302dbac46a1ab&role.id=ea3b2f1eb283418b8823cb4fff80edba => generated 608 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:23.990942 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff5ef0-a0e0-4535-a190-c09de48a72ce None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:24.003144 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc8e49-c575-4758-857a-8bd37a550b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:24.005356 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc8e49-c575-4758-857a-8bd37a550b25 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:24.007460 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc8e49-c575-4758-857a-8bd37a550b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:24.007948 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc8e49-c575-4758-857a-8bd37a550b25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:24.009218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc8e49-c575-4758-857a-8bd37a550b25 None admin] RBAC: Adding query filter params (role.id=ea3b2f1eb283418b8823cb4fff80edba, scope.project.id=70ac515938a24c45ae6302dbac46a1ab, user.id=453297dac64046d19fc1c331d2d85abc) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:24.009702 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc8e49-c575-4758-857a-8bd37a550b25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:24.011525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc8e49-c575-4758-857a-8bd37a550b25 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:24.020735 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 345/675] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:59:23 2022] GET /identity/v3/role_assignments?user.id=453297dac64046d19fc1c331d2d85abc&scope.project.id=70ac515938a24c45ae6302dbac46a1ab&role.id=ea3b2f1eb283418b8823cb4fff80edba => generated 608 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:24.021903 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc8e49-c575-4758-857a-8bd37a550b25 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:24.125564 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:24.246448 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff5ef0-a0e0-4535-a190-c09de48a72ce None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:24.288747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc8e49-c575-4758-857a-8bd37a550b25 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:24.321621 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91870e34-9521-4fb8-b9cd-61276c682a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:24.322249 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91870e34-9521-4fb8-b9cd-61276c682a6b None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:24.322526 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91870e34-9521-4fb8-b9cd-61276c682a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:24.322805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91870e34-9521-4fb8-b9cd-61276c682a6b None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:24.323456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91870e34-9521-4fb8-b9cd-61276c682a6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:24.323832 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91870e34-9521-4fb8-b9cd-61276c682a6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:24.325975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91870e34-9521-4fb8-b9cd-61276c682a6b None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:24.342473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 331/676] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:24.353038 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91870e34-9521-4fb8-b9cd-61276c682a6b None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:24.369904 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35528da-b172-4d7f-85f8-3d1f06620c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:24.370852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35528da-b172-4d7f-85f8-3d1f06620c7a None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:24.371366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35528da-b172-4d7f-85f8-3d1f06620c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:24.371820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35528da-b172-4d7f-85f8-3d1f06620c7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:24.436243 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:24.511325 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35528da-b172-4d7f-85f8-3d1f06620c7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:24.513191 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35528da-b172-4d7f-85f8-3d1f06620c7a None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:24.539562 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91870e34-9521-4fb8-b9cd-61276c682a6b None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:24.598288 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c5966-d0cb-4090-b97b-a9d04bb54cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:24.599347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c5966-d0cb-4090-b97b-a9d04bb54cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:24.600082 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c5966-d0cb-4090-b97b-a9d04bb54cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:24.600542 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c5966-d0cb-4090-b97b-a9d04bb54cf7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:24.615095 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c5966-d0cb-4090-b97b-a9d04bb54cf7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:24.617439 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c5966-d0cb-4090-b97b-a9d04bb54cf7 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:24.629663 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:24.635903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-832c5966-d0cb-4090-b97b-a9d04bb54cf7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 14:59:24.635903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:24.635903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:24.635903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:24.635903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:24.635903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:24.635903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:24.635903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:24.635903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:24.635903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:24.635903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:24.635903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:24.635903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:59:24.635903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:59:24.635903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:59:24.635903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:59:24.637593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:24.637593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:24.637593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:59:24.637593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:24.637593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:59:24.637593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:24.637593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:24.637593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:24.637593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:24.637593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:24.637593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:24.637593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:24.637593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:24.637593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:24.637593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:24.637593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:24.637593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:24.637593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:24.638759 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:24.638759 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:24.638759 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:59:24.638759 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:59:24.638759 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:59:24.638759 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:59:24.638759 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:59:24.638759 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:59:24.638759 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 14:59:24.638759 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:24.642265 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 332/677] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 14:59:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:24.650260 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c5966-d0cb-4090-b97b-a9d04bb54cf7 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:24.714262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 346/678] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:59:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 693 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:59:24.724521 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35528da-b172-4d7f-85f8-3d1f06620c7a None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:24.848220 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c5966-d0cb-4090-b97b-a9d04bb54cf7 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:24.878906 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:24.897336 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ac673-ac70-451c-ba6b-aa0e570256ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:24.898112 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ac673-ac70-451c-ba6b-aa0e570256ae None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:24.898402 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ac673-ac70-451c-ba6b-aa0e570256ae None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:24.898670 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ac673-ac70-451c-ba6b-aa0e570256ae None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:24.899520 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ac673-ac70-451c-ba6b-aa0e570256ae None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:24.899824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ac673-ac70-451c-ba6b-aa0e570256ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:24.901486 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ac673-ac70-451c-ba6b-aa0e570256ae None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:24.925442 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35528da-b172-4d7f-85f8-3d1f06620c7a None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:24.931548 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 333/679] 10.222.0.44 () {56 vars in 1135 bytes} [Sun Sep 11 14:59:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:24.937356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ac673-ac70-451c-ba6b-aa0e570256ae None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:24.974035 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f1ec1-8e10-4775-a7f9-75bd9d88cfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:24.974686 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f1ec1-8e10-4775-a7f9-75bd9d88cfa7 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:24.974958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f1ec1-8e10-4775-a7f9-75bd9d88cfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:24.975226 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f1ec1-8e10-4775-a7f9-75bd9d88cfa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:25.010324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f1ec1-8e10-4775-a7f9-75bd9d88cfa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:25.013226 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f1ec1-8e10-4775-a7f9-75bd9d88cfa7 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:25.068674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-2e9f1ec1-8e10-4775-a7f9-75bd9d88cfa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:25.068674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:25.068674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:25.068674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:25.068674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:25.068674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:25.068674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:25.068674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:25.068674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:25.068674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:25.068674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:25.068674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:25.068674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:25.068674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:25.068674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:25.068674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:25.069727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:25.069727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:25.069727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:25.069727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:25.069727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:25.069727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:25.069727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:25.069727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:25.069727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:25.069727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:25.069727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:25.069727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:25.069727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:25.069727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:25.069727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:25.069727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:25.069727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:25.070607 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:25.070607 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:25.070607 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:25.081794 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 347/680] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:25.088659 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f1ec1-8e10-4775-a7f9-75bd9d88cfa7 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:25.102775 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:25.200098 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ac673-ac70-451c-ba6b-aa0e570256ae None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:25.326014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27b83a-2bcf-4589-b770-8c9cb043ed5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:25.326670 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27b83a-2bcf-4589-b770-8c9cb043ed5c None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:25.326948 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27b83a-2bcf-4589-b770-8c9cb043ed5c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:25.327219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27b83a-2bcf-4589-b770-8c9cb043ed5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:25.385577 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f1ec1-8e10-4775-a7f9-75bd9d88cfa7 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:25.458340 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7eb88-6c71-407d-811b-1362cae72c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:25.459352 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7eb88-6c71-407d-811b-1362cae72c8a None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:25.459816 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7eb88-6c71-407d-811b-1362cae72c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:25.460286 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7eb88-6c71-407d-811b-1362cae72c8a None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:25.461723 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc7eb88-6c71-407d-811b-1362cae72c8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:25.462048 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc7eb88-6c71-407d-811b-1362cae72c8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:25.464092 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc7eb88-6c71-407d-811b-1362cae72c8a None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:25.484380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 348/681] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:25.490484 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:25.494343 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7eb88-6c71-407d-811b-1362cae72c8a None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:25.546621 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27b83a-2bcf-4589-b770-8c9cb043ed5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:25.557177 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27b83a-2bcf-4589-b770-8c9cb043ed5c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:25.563545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:25.607422 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7eb88-6c71-407d-811b-1362cae72c8a None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:25.644147 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43214be2-c9dd-4843-907a-f93e40d1ca85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:25.644750 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43214be2-c9dd-4843-907a-f93e40d1ca85 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:25.645024 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43214be2-c9dd-4843-907a-f93e40d1ca85 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:25.645319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43214be2-c9dd-4843-907a-f93e40d1ca85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:25.653825 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43214be2-c9dd-4843-907a-f93e40d1ca85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:25.656128 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43214be2-c9dd-4843-907a-f93e40d1ca85 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:25.676822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-43214be2-c9dd-4843-907a-f93e40d1ca85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:25.676822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:25.676822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:25.676822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:25.676822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:25.676822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:25.676822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:25.676822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:25.676822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:25.676822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:25.676822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:25.676822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:25.676822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:59:25.676822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:59:25.676822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:59:25.677836 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:59:25.677836 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:25.677836 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:25.677836 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:25.677836 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:25.677836 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:25.677836 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:25.677836 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:25.677836 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:25.677836 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:25.677836 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:25.677836 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:25.677836 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:25.677836 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:25.677836 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:25.677836 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:25.677836 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:25.678727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:59:25.678727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:59:25.678727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:59:25.678727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:59:25.678727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:59:25.678727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:59:25.678727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:25.678727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:25.679692 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 349/682] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:59:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:25.681463 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f8f81-3284-4685-8c6e-c35ec7d270b3 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:25.681949 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f8f81-3284-4685-8c6e-c35ec7d270b3 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:25.682428 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f8f81-3284-4685-8c6e-c35ec7d270b3 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:25.688909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 350/683] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:25.688909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f8f81-3284-4685-8c6e-c35ec7d270b3 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:25.709638 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 334/684] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:59:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 773 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 14:59:25.712385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c354c-4a23-4c1a-8368-a48b14996c39 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:25.712900 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c354c-4a23-4c1a-8368-a48b14996c39 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:25.713429 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c354c-4a23-4c1a-8368-a48b14996c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:25.738513 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-a44c354c-4a23-4c1a-8368-a48b14996c39 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:25.739586 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-a44c354c-4a23-4c1a-8368-a48b14996c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:25.847490 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 335/685] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:59:25.848710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c354c-4a23-4c1a-8368-a48b14996c39 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:25.853195 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:25.861781 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f8f81-3284-4685-8c6e-c35ec7d270b3 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:25.898243 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:25.920918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbfcfb-3349-4316-b0d5-740f9238f983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:25.922672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbfcfb-3349-4316-b0d5-740f9238f983 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:25.923732 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbfcfb-3349-4316-b0d5-740f9238f983 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:25.924513 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbfcfb-3349-4316-b0d5-740f9238f983 None admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:25.926208 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bbfcfb-3349-4316-b0d5-740f9238f983 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:25.927283 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bbfcfb-3349-4316-b0d5-740f9238f983 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:25.930372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bbfcfb-3349-4316-b0d5-740f9238f983 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:25.956023 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 351/686] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 14:59:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:25.965265 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbfcfb-3349-4316-b0d5-740f9238f983 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:25.970228 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c354c-4a23-4c1a-8368-a48b14996c39 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:26.015318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef080e4-66ae-40e6-8503-c9e535b70a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:26.015974 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef080e4-66ae-40e6-8503-c9e535b70a17 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:26.016254 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef080e4-66ae-40e6-8503-c9e535b70a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:26.016521 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef080e4-66ae-40e6-8503-c9e535b70a17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:26.030753 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef080e4-66ae-40e6-8503-c9e535b70a17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:26.032268 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef080e4-66ae-40e6-8503-c9e535b70a17 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:26.051757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-7ef080e4-66ae-40e6-8503-c9e535b70a17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:26.051757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:26.051757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:26.051757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:26.051757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:26.051757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:26.051757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:26.051757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:26.051757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:26.051757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:26.051757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:26.051757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:26.051757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:26.051757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:26.051757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:26.051757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:26.052802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:26.052802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:26.052802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:26.052802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:26.052802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:26.052802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:26.052802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:26.052802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:26.052802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:26.052802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:26.052802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:26.052802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:26.052802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:26.052802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:26.052802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:26.052802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:26.052802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:26.053745 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:26.053745 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:26.053745 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:26.053745 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 336/687] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:26.059390 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef080e4-66ae-40e6-8503-c9e535b70a17 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:26.137563 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:26.208102 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef080e4-66ae-40e6-8503-c9e535b70a17 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:26.264908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbfcfb-3349-4316-b0d5-740f9238f983 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:26.284045 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd364d4-3128-4b7f-a35d-051688abd6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:26.285077 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd364d4-3128-4b7f-a35d-051688abd6f4 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:26.285595 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd364d4-3128-4b7f-a35d-051688abd6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:26.286232 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd364d4-3128-4b7f-a35d-051688abd6f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:26.287225 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd364d4-3128-4b7f-a35d-051688abd6f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:26.287739 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd364d4-3128-4b7f-a35d-051688abd6f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:26.289478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd364d4-3128-4b7f-a35d-051688abd6f4 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:26.305098 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 337/688] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:26.306507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd364d4-3128-4b7f-a35d-051688abd6f4 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:26.317254 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0914ee3-5c65-457b-9191-ff71381866d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:26.317912 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0914ee3-5c65-457b-9191-ff71381866d4 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:26.318234 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0914ee3-5c65-457b-9191-ff71381866d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:26.319171 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0914ee3-5c65-457b-9191-ff71381866d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:26.320018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0914ee3-5c65-457b-9191-ff71381866d4 None admin] RBAC: Adding query filter params (role.id=ea3b2f1eb283418b8823cb4fff80edba, scope.project.id=70ac515938a24c45ae6302dbac46a1ab, user.id=8b97e2542db54900a99f6f44b78a5e24) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:26.320336 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0914ee3-5c65-457b-9191-ff71381866d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:26.322489 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0914ee3-5c65-457b-9191-ff71381866d4 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:26.330753 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 352/689] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:59:25 2022] GET /identity/v3/role_assignments?user.id=8b97e2542db54900a99f6f44b78a5e24&scope.project.id=70ac515938a24c45ae6302dbac46a1ab&role.id=ea3b2f1eb283418b8823cb4fff80edba => generated 608 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:26.331665 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0914ee3-5c65-457b-9191-ff71381866d4 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:26.398004 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:26.435945 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd364d4-3128-4b7f-a35d-051688abd6f4 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:26.458365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:26.474666 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0914ee3-5c65-457b-9191-ff71381866d4 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:26.480750 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e1523-0ed5-43cb-9d4c-d00bb70d5cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:26.481658 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e1523-0ed5-43cb-9d4c-d00bb70d5cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:26.482034 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e1523-0ed5-43cb-9d4c-d00bb70d5cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:26.482396 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e1523-0ed5-43cb-9d4c-d00bb70d5cc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:26.495918 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887e1523-0ed5-43cb-9d4c-d00bb70d5cc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:26.497992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887e1523-0ed5-43cb-9d4c-d00bb70d5cc4 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:26.515338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-887e1523-0ed5-43cb-9d4c-d00bb70d5cc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:26.515338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:26.515338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:26.515338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:26.515338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:26.515338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:26.515338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:26.515338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:26.515338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:26.515338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:26.515338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:26.515338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:26.515338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:26.515338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:26.515338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:26.515338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:26.516379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:26.516379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:26.516379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:26.516379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:26.516379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:26.516379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:26.516379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:26.516379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:26.516379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:26.516379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:26.516379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:26.516379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:26.516379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:26.516379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:26.516379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:26.516379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:26.516379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:26.517318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:26.517318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:26.517318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:26.517318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 338/690] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:26.521981 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fdd76d-27e1-45e9-96d5-7240e88dbff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:26.522929 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdd76d-27e1-45e9-96d5-7240e88dbff6 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:26.523375 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e1523-0ed5-43cb-9d4c-d00bb70d5cc4 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:26.523606 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdd76d-27e1-45e9-96d5-7240e88dbff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:26.524050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdd76d-27e1-45e9-96d5-7240e88dbff6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:26.532633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fdd76d-27e1-45e9-96d5-7240e88dbff6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:26.534638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fdd76d-27e1-45e9-96d5-7240e88dbff6 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:26.548193 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-d9fdd76d-27e1-45e9-96d5-7240e88dbff6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:26.548193 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:26.548193 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:26.548193 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:26.548193 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:26.548193 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:26.548193 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:26.548193 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:26.548193 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:26.548193 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:26.548193 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:26.548193 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:26.548193 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:59:26.548193 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:59:26.548193 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:59:26.549200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:59:26.549200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:26.549200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:26.549200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:26.549200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:26.549200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:26.549200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:26.549200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:26.549200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:26.549200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:26.549200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:26.549200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:26.549200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:26.549200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:26.549200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:26.549200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:26.549200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:26.550187 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:59:26.550187 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:59:26.550187 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:59:26.550187 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:59:26.550187 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:59:26.550187 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:59:26.550187 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:26.550187 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:26.552881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 353/691] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:59:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:26.559917 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fdd76d-27e1-45e9-96d5-7240e88dbff6 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:26.752403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fdd76d-27e1-45e9-96d5-7240e88dbff6 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:26.766633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:26.768119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e1523-0ed5-43cb-9d4c-d00bb70d5cc4 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:26.818627 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8658d9b3-57e2-4f02-ac7c-0af3a0e37b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:26.819267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658d9b3-57e2-4f02-ac7c-0af3a0e37b0c None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:26.819545 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658d9b3-57e2-4f02-ac7c-0af3a0e37b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:26.819814 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658d9b3-57e2-4f02-ac7c-0af3a0e37b0c None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:26.820481 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8658d9b3-57e2-4f02-ac7c-0af3a0e37b0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:26.820785 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8658d9b3-57e2-4f02-ac7c-0af3a0e37b0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:26.822443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8658d9b3-57e2-4f02-ac7c-0af3a0e37b0c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:26.829913 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654da545-17b9-4d0b-a526-11d4f346f45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:26.830902 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654da545-17b9-4d0b-a526-11d4f346f45f None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:26.832098 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654da545-17b9-4d0b-a526-11d4f346f45f None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:26.832632 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654da545-17b9-4d0b-a526-11d4f346f45f None admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:26.834073 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654da545-17b9-4d0b-a526-11d4f346f45f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:26.834757 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654da545-17b9-4d0b-a526-11d4f346f45f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:26.836490 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654da545-17b9-4d0b-a526-11d4f346f45f None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:26.837363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 339/692] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:26.838989 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd470a4-9ea1-4cee-9bde-5dd8f4f5f31d None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:26.839435 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd470a4-9ea1-4cee-9bde-5dd8f4f5f31d None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:26.839798 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd470a4-9ea1-4cee-9bde-5dd8f4f5f31d None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:26.841781 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 340/693] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:26.842700 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd470a4-9ea1-4cee-9bde-5dd8f4f5f31d None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:26.860430 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 354/694] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 14:59:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:26.870018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35f1af-39d9-437b-a04c-0c201bf3ebcd None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:26.870018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35f1af-39d9-437b-a04c-0c201bf3ebcd None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:26.870018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35f1af-39d9-437b-a04c-0c201bf3ebcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:26.886254 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:26.906639 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-7e35f1af-39d9-437b-a04c-0c201bf3ebcd None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:26.908700 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-7e35f1af-39d9-437b-a04c-0c201bf3ebcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:26.987029 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd470a4-9ea1-4cee-9bde-5dd8f4f5f31d None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:27.004624 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 355/695] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:59:27.009347 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35f1af-39d9-437b-a04c-0c201bf3ebcd None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:27.051247 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be3d6c-9d40-46f4-b864-bf4a5a24ad3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:27.052115 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be3d6c-9d40-46f4-b864-bf4a5a24ad3c None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:27.052679 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be3d6c-9d40-46f4-b864-bf4a5a24ad3c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:27.053298 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be3d6c-9d40-46f4-b864-bf4a5a24ad3c None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:27.054709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16be3d6c-9d40-46f4-b864-bf4a5a24ad3c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:27.057785 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16be3d6c-9d40-46f4-b864-bf4a5a24ad3c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:27.105947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 341/696] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 14:59:26 2022] POST /identity/v3/users => generated 254 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:59:27.108649 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4445e4-a17e-4c24-ba50-835b0ef36bf8 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:27.109531 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4445e4-a17e-4c24-ba50-835b0ef36bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:27.110246 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4445e4-a17e-4c24-ba50-835b0ef36bf8 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:27.116656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 342/697] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:27 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:27.127414 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:27.134935 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35f1af-39d9-437b-a04c-0c201bf3ebcd None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:27.135962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b58f53-cc22-47a0-8373-061cd5125c0f None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:27.136643 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b58f53-cc22-47a0-8373-061cd5125c0f None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:27.137365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b58f53-cc22-47a0-8373-061cd5125c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:27.190220 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-52b58f53-cc22-47a0-8373-061cd5125c0f None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:27.191944 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-52b58f53-cc22-47a0-8373-061cd5125c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:27.194008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeaa959-0912-4a9c-b7b8-b842feadfd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:27.194649 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeaa959-0912-4a9c-b7b8-b842feadfd69 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:27.194950 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeaa959-0912-4a9c-b7b8-b842feadfd69 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:27.200135 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeaa959-0912-4a9c-b7b8-b842feadfd69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:27.200135 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeaa959-0912-4a9c-b7b8-b842feadfd69 None admin] RBAC: Adding query filter params (role.id=ea3b2f1eb283418b8823cb4fff80edba, scope.project.id=70ac515938a24c45ae6302dbac46a1ab, user.id=05aa5e40ee4542719657bc03b2015e72) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:27.200135 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeaa959-0912-4a9c-b7b8-b842feadfd69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:27.200135 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeaa959-0912-4a9c-b7b8-b842feadfd69 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:27.208128 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 356/698] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:59:27 2022] GET /identity/v3/role_assignments?user.id=05aa5e40ee4542719657bc03b2015e72&scope.project.id=70ac515938a24c45ae6302dbac46a1ab&role.id=ea3b2f1eb283418b8823cb4fff80edba => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:27.309018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeaa959-0912-4a9c-b7b8-b842feadfd69 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:27.328570 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 343/699] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:59:27.441823 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeaa959-0912-4a9c-b7b8-b842feadfd69 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:27.484076 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f70b9-0355-4625-adfe-cd0a9d5b9e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:27.484746 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f70b9-0355-4625-adfe-cd0a9d5b9e2d None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:27.485080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f70b9-0355-4625-adfe-cd0a9d5b9e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:27.485379 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f70b9-0355-4625-adfe-cd0a9d5b9e2d None admin] PATH_INFO: `/v3/services/network` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:27.485985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851f70b9-0355-4625-adfe-cd0a9d5b9e2d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:27.487417 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851f70b9-0355-4625-adfe-cd0a9d5b9e2d None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:27.499689 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-851f70b9-0355-4625-adfe-cd0a9d5b9e2d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 14:59:27.499689 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:27.499689 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:59:27.499689 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:59:27.499689 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:59:27.499689 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:59:27.499689 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:59:27.499689 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:59:27.499689 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 14:59:27.499689 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:27.499689 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:59:27.499689 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:27.499689 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:27.499689 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:27.499689 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:27.499689 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:27.499689 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:27.501654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:27.501654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:27.501654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:27.501654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:27.501654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:27.501654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:27.501654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:59:27.501654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:59:27.501654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:59:27.501654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:59:27.501654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:27.501654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:27.501654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:27.501654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:27.501654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:27.501654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:27.503959 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:27.503959 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:27.503959 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:27.503959 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:27.503959 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:27.503959 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:27.503959 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:27.503959 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:27.503959 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:27.503959 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:27.503959 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:59:27.503959 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:59:27.503959 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 14:59:27.503959 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:27.503959 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 357/700] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:59:27 2022] GET /identity/v3/services/network => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:59:27.509321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b58f53-cc22-47a0-8373-061cd5125c0f None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:27.533000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:27.646789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b58f53-cc22-47a0-8373-061cd5125c0f None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:27.710632 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab48ee-dc76-44d7-a2a7-9f5b64f5f245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:27.711598 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab48ee-dc76-44d7-a2a7-9f5b64f5f245 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:27.711883 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab48ee-dc76-44d7-a2a7-9f5b64f5f245 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:27.712343 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab48ee-dc76-44d7-a2a7-9f5b64f5f245 None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:27.713568 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab48ee-dc76-44d7-a2a7-9f5b64f5f245 None admin] RBAC: Adding query filter params (name=network) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:27.714113 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab48ee-dc76-44d7-a2a7-9f5b64f5f245 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:27.716874 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab48ee-dc76-44d7-a2a7-9f5b64f5f245 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:27.726035 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 344/701] 10.222.0.44 () {56 vars in 1112 bytes} [Sun Sep 11 14:59:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:27.732351 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f70b9-0355-4625-adfe-cd0a9d5b9e2d None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:27.767101 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab48ee-dc76-44d7-a2a7-9f5b64f5f245 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:27.788028 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:27.896976 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:27.931653 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab48ee-dc76-44d7-a2a7-9f5b64f5f245 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:27.959247 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f70b9-0355-4625-adfe-cd0a9d5b9e2d None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:27.979308 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7302d6-3502-4773-b2cc-df62a69a7f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:27.980205 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7302d6-3502-4773-b2cc-df62a69a7f48 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:27.980579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7302d6-3502-4773-b2cc-df62a69a7f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:27.980945 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7302d6-3502-4773-b2cc-df62a69a7f48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:27.996315 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7302d6-3502-4773-b2cc-df62a69a7f48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:59:28.002591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-9a7302d6-3502-4773-b2cc-df62a69a7f48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:59:28.002591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:28.002591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:28.002591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:28.002591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:28.002591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:28.002591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:28.002591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:28.002591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:28.002591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:28.002591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:28.002591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:28.002591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:59:28.002591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:59:28.002591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:59:28.002591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:59:28.004374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:59:28.004374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:59:28.004374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:59:28.004374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:59:28.004374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:28.004374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:28.004374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:28.004374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:28.004374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:28.004374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:28.004374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:28.004374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:28.004374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:28.004374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:28.004374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:28.004374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:28.006683 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:28.006683 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:28.006683 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:28.006683 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:28.006683 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:59:28.006683 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:59:28.006683 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:59:28.006683 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:59:28.006683 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:59:28.006683 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:59:28.006683 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:59:28.006683 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:28.006683 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 345/702] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 14:59:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:28.011198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7302d6-3502-4773-b2cc-df62a69a7f48 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:28.058342 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3e12e-0f2c-4741-a521-36a2b830ade6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:28.058709 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3e12e-0f2c-4741-a521-36a2b830ade6 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:28.059021 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3e12e-0f2c-4741-a521-36a2b830ade6 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:28.059331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3e12e-0f2c-4741-a521-36a2b830ade6 None admin] PATH_INFO: `/v3/services` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:28.059983 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e3e12e-0f2c-4741-a521-36a2b830ade6 None admin] RBAC: Adding query filter params (type=network) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:28.060281 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e3e12e-0f2c-4741-a521-36a2b830ade6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:28.061822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e3e12e-0f2c-4741-a521-36a2b830ade6 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:28.076804 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 358/703] 10.222.0.44 () {56 vars in 1112 bytes} [Sun Sep 11 14:59:27 2022] GET /identity/v3/services?type=network => generated 124 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:28.182129 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7302d6-3502-4773-b2cc-df62a69a7f48 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:28.240599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804b417-a29c-474e-a411-02fb2e86cfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:28.242253 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804b417-a29c-474e-a411-02fb2e86cfaa None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:28.243111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804b417-a29c-474e-a411-02fb2e86cfaa None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:28.243786 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804b417-a29c-474e-a411-02fb2e86cfaa None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:28.245431 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e804b417-a29c-474e-a411-02fb2e86cfaa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:28.246249 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e804b417-a29c-474e-a411-02fb2e86cfaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:28.253592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e804b417-a29c-474e-a411-02fb2e86cfaa None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:28.266646 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:28.272157 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 346/704] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:59:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:59:28.287078 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3e12e-0f2c-4741-a521-36a2b830ade6 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:28.519918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:28.545940 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3e12e-0f2c-4741-a521-36a2b830ade6 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:28.586213 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86612250-473d-4691-961c-16f561997fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:28.586869 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86612250-473d-4691-961c-16f561997fdf None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:28.587143 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86612250-473d-4691-961c-16f561997fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:28.587411 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86612250-473d-4691-961c-16f561997fdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:28.599706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86612250-473d-4691-961c-16f561997fdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:28.601213 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86612250-473d-4691-961c-16f561997fdf None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:28.617524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-86612250-473d-4691-961c-16f561997fdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:28.617524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:28.617524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:28.617524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:28.617524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:28.617524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:28.617524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:28.617524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:28.617524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:28.617524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:28.617524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:28.617524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:28.617524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:28.617524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:28.617524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:28.617524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:28.618545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:28.618545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:28.618545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:28.618545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:28.618545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:28.618545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:28.618545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:28.618545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:28.618545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:28.618545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:28.618545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:28.618545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:28.618545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:28.618545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:28.618545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:28.618545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:28.618545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:28.620551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:28.620551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:28.620551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:28.620551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 359/705] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:28.626719 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804b417-a29c-474e-a411-02fb2e86cfaa None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:28.816276 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804b417-a29c-474e-a411-02fb2e86cfaa None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:28.869948 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d4a8f-42d5-469e-97ae-e65003ecc68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:28.870638 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d4a8f-42d5-469e-97ae-e65003ecc68c None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:28.870920 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d4a8f-42d5-469e-97ae-e65003ecc68c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:28.871190 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d4a8f-42d5-469e-97ae-e65003ecc68c None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:28.871855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d4a8f-42d5-469e-97ae-e65003ecc68c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:28.872149 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d4a8f-42d5-469e-97ae-e65003ecc68c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:28.873801 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1d4a8f-42d5-469e-97ae-e65003ecc68c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:28.887578 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 347/706] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:59:28.893369 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86612250-473d-4691-961c-16f561997fdf None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:29.114903 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86612250-473d-4691-961c-16f561997fdf None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:29.183977 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59669a-39f9-4815-a2ab-62d9385ed0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:29.185056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59669a-39f9-4815-a2ab-62d9385ed0d8 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:29.185572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59669a-39f9-4815-a2ab-62d9385ed0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:29.186051 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59669a-39f9-4815-a2ab-62d9385ed0d8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:29.200863 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59669a-39f9-4815-a2ab-62d9385ed0d8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:29.203654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59669a-39f9-4815-a2ab-62d9385ed0d8 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:29.213788 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:29.237862 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-fa59669a-39f9-4815-a2ab-62d9385ed0d8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:59:29.237862 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:29.237862 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:29.237862 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:29.237862 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:29.237862 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:29.237862 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:29.237862 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:29.237862 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:29.237862 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:29.237862 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:29.237862 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:29.237862 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:59:29.237862 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:59:29.237862 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:59:29.237862 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:59:29.239018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:29.239018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:29.239018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:59:29.239018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:29.239018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:59:29.239018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:29.239018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:29.239018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:29.239018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:29.239018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:29.239018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:29.239018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:29.239018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:29.239018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:29.239018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:29.239018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:29.239018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:29.239018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:29.239970 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:29.239970 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:29.239970 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:59:29.239970 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:59:29.239970 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:59:29.239970 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:59:29.239970 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:59:29.239970 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:59:29.239970 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:59:29.239970 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:29.243336 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 360/707] 10.222.0.44 () {56 vars in 1093 bytes} [Sun Sep 11 14:59:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:29.252182 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d4a8f-42d5-469e-97ae-e65003ecc68c None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:29.374270 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:29.477044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d4a8f-42d5-469e-97ae-e65003ecc68c None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:29.557539 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621ce90-7dae-491a-bdf7-a00ddf71e6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:29.558601 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621ce90-7dae-491a-bdf7-a00ddf71e6c6 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:29.559076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621ce90-7dae-491a-bdf7-a00ddf71e6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:29.559554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621ce90-7dae-491a-bdf7-a00ddf71e6c6 None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:29.560850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7621ce90-7dae-491a-bdf7-a00ddf71e6c6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:29.561414 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7621ce90-7dae-491a-bdf7-a00ddf71e6c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:29.564035 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7621ce90-7dae-491a-bdf7-a00ddf71e6c6 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:29.614443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 348/708] 10.222.0.44 () {56 vars in 1133 bytes} [Sun Sep 11 14:59:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:29.627090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59669a-39f9-4815-a2ab-62d9385ed0d8 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:29.848418 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59669a-39f9-4815-a2ab-62d9385ed0d8 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:29.927258 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78f91a-cbd6-4df9-b5f3-8e4f9e34f7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:29.928222 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78f91a-cbd6-4df9-b5f3-8e4f9e34f7f6 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:29.928704 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78f91a-cbd6-4df9-b5f3-8e4f9e34f7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:29.929217 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78f91a-cbd6-4df9-b5f3-8e4f9e34f7f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:30.008318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea72084-c90c-4c66-b5db-cf4cbff41c2a None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:30.008318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea72084-c90c-4c66-b5db-cf4cbff41c2a None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:30.008318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea72084-c90c-4c66-b5db-cf4cbff41c2a None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:30.008318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 349/709] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:59:30.043579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca208e-db58-4d67-8726-c1e32314fb8d None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:30.044465 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca208e-db58-4d67-8726-c1e32314fb8d None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:30.045158 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca208e-db58-4d67-8726-c1e32314fb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:30.077698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:30.085190 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-4dca208e-db58-4d67-8726-c1e32314fb8d None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:30.086819 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-4dca208e-db58-4d67-8726-c1e32314fb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:30.092266 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78f91a-cbd6-4df9-b5f3-8e4f9e34f7f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:30.094929 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78f91a-cbd6-4df9-b5f3-8e4f9e34f7f6 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:30.172625 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 350/710] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:59:30.284017 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 361/711] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:59:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 658 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:59:30.293726 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca208e-db58-4d67-8726-c1e32314fb8d None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:30.381684 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:30.456445 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca208e-db58-4d67-8726-c1e32314fb8d None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:30.509170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a464902-85b2-4547-a131-d2e9ae5a2062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:30.509812 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a464902-85b2-4547-a131-d2e9ae5a2062 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:30.510081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a464902-85b2-4547-a131-d2e9ae5a2062 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:30.510344 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a464902-85b2-4547-a131-d2e9ae5a2062 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:30.524465 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a464902-85b2-4547-a131-d2e9ae5a2062 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:30.525959 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a464902-85b2-4547-a131-d2e9ae5a2062 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:30.545503 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-7a464902-85b2-4547-a131-d2e9ae5a2062 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:30.545503 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:30.545503 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:30.545503 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:30.545503 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:30.545503 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:30.545503 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:30.545503 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:30.545503 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:30.545503 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:30.545503 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:30.545503 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:30.545503 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:30.545503 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:30.545503 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:30.545503 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:30.547223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:30.547223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:30.547223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:30.547223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:30.547223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:30.547223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:30.547223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:30.547223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:30.547223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:30.547223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:30.547223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:30.547223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:30.547223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:30.547223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:30.547223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:30.547223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:30.547223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:30.548851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:30.548851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:30.548851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:30.548851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 351/712] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:30.561191 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78f91a-cbd6-4df9-b5f3-8e4f9e34f7f6 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:30.730801 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78f91a-cbd6-4df9-b5f3-8e4f9e34f7f6 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:30.732780 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:30.772809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e9bfe-7f86-4587-bd0a-759fabd00c35 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:30.773480 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e9bfe-7f86-4587-bd0a-759fabd00c35 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:30.773951 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e9bfe-7f86-4587-bd0a-759fabd00c35 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:30.774211 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fec9a02-2a9f-489c-aa67-a651b916e205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:30.774863 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec9a02-2a9f-489c-aa67-a651b916e205 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:30.775143 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec9a02-2a9f-489c-aa67-a651b916e205 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:30.775495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec9a02-2a9f-489c-aa67-a651b916e205 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:30.776199 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec9a02-2a9f-489c-aa67-a651b916e205 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:30.776504 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec9a02-2a9f-489c-aa67-a651b916e205 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:30.777037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 352/713] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:59:30.780362 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec9a02-2a9f-489c-aa67-a651b916e205 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:30.785126 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fcf10-265c-4ddd-bb29-85a66720f631 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:30.785808 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fcf10-265c-4ddd-bb29-85a66720f631 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:30.786282 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fcf10-265c-4ddd-bb29-85a66720f631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:30.801549 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 362/714] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:59:30.813883 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fec9a02-2a9f-489c-aa67-a651b916e205 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:30.824736 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-e91fcf10-265c-4ddd-bb29-85a66720f631 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:30.827400 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-e91fcf10-265c-4ddd-bb29-85a66720f631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:30.944984 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 353/715] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 14:59:30.947331 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91fcf10-265c-4ddd-bb29-85a66720f631 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:30.966369 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fec9a02-2a9f-489c-aa67-a651b916e205 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:31.006217 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:31.009325 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a3069-c5f6-47f2-b112-759e5810f34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:31.010106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a3069-c5f6-47f2-b112-759e5810f34c None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:31.010599 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a3069-c5f6-47f2-b112-759e5810f34c None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:31.010874 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a3069-c5f6-47f2-b112-759e5810f34c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:31.021614 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a3069-c5f6-47f2-b112-759e5810f34c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:31.025807 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a3069-c5f6-47f2-b112-759e5810f34c None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:31.036220 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-b68a3069-c5f6-47f2-b112-759e5810f34c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:31.036220 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:31.036220 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:31.036220 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:31.036220 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:31.036220 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:31.036220 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:31.036220 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:31.036220 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:31.036220 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:31.036220 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:31.036220 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:31.036220 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:59:31.036220 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:59:31.036220 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:59:31.038148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:59:31.038148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:31.038148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:31.038148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:31.038148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:31.038148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:31.038148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:31.038148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:31.038148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:31.038148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:31.038148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:31.038148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:31.038148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:31.038148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:31.038148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:31.038148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:31.038148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:31.039814 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:59:31.039814 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:59:31.039814 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:59:31.039814 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:59:31.039814 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:59:31.039814 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:59:31.039814 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:31.039814 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:31.042582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 363/716] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:59:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:31.044760 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a3069-c5f6-47f2-b112-759e5810f34c None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:31.089644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:31.143410 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91fcf10-265c-4ddd-bb29-85a66720f631 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:31.167001 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a3069-c5f6-47f2-b112-759e5810f34c None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:31.203750 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06d7c4-ea75-43c0-a14d-84d21c97c567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:31.204473 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288cc260-d962-46f3-9efa-9360e1b52294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:31.204663 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06d7c4-ea75-43c0-a14d-84d21c97c567 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:31.204927 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06d7c4-ea75-43c0-a14d-84d21c97c567 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:31.205073 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288cc260-d962-46f3-9efa-9360e1b52294 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:31.205443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06d7c4-ea75-43c0-a14d-84d21c97c567 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:31.205602 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288cc260-d962-46f3-9efa-9360e1b52294 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:31.205731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288cc260-d962-46f3-9efa-9360e1b52294 None admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:31.205820 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db06d7c4-ea75-43c0-a14d-84d21c97c567 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:31.209166 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288cc260-d962-46f3-9efa-9360e1b52294 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:31.209166 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288cc260-d962-46f3-9efa-9360e1b52294 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:31.209535 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db06d7c4-ea75-43c0-a14d-84d21c97c567 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:31.209685 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288cc260-d962-46f3-9efa-9360e1b52294 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:31.221523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-db06d7c4-ea75-43c0-a14d-84d21c97c567 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 14:59:31.221523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:31.221523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:59:31.221523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:59:31.221523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:59:31.221523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:59:31.221523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:59:31.221523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:59:31.221523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 14:59:31.221523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:31.221523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:59:31.221523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:31.221523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:31.221523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:31.221523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:31.221523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:31.221523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:31.223312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:31.223312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:31.223312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:31.223312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:31.223312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:31.223312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:31.223312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:59:31.223312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:59:31.223312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:59:31.223312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:59:31.223312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:31.223312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:31.223312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:31.223312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:31.223312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:31.223312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:31.224534 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:31.224534 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:31.224534 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:31.224534 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:31.224534 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:31.224534 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:31.224534 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:31.224534 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:31.224534 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:31.224534 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:31.224534 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:59:31.224534 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:59:31.224534 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 14:59:31.224534 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:31.227296 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 364/717] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 14:59:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:31.227296 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73eacae-1fdb-4fb1-b8a0-b9a0f8be4df9 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:31.227296 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73eacae-1fdb-4fb1-b8a0-b9a0f8be4df9 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:31.227296 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73eacae-1fdb-4fb1-b8a0-b9a0f8be4df9 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:31.228142 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 354/719] 10.222.0.44 () {56 vars in 1129 bytes} [Sun Sep 11 14:59:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:31.237190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 365/719] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:31.237190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73eacae-1fdb-4fb1-b8a0-b9a0f8be4df9 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:31.238060 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224db38e-87bc-4879-9057-e348884150ec None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:31.238060 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224db38e-87bc-4879-9057-e348884150ec None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:31.238060 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224db38e-87bc-4879-9057-e348884150ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:31.259561 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-224db38e-87bc-4879-9057-e348884150ec None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:31.260794 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-224db38e-87bc-4879-9057-e348884150ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:31.335203 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:31.347641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 355/720] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:59:31.349174 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224db38e-87bc-4879-9057-e348884150ec None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:31.373092 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73eacae-1fdb-4fb1-b8a0-b9a0f8be4df9 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:31.378354 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:31.418234 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4397b-0b40-4efe-8c0c-654694dd3082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:31.419292 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4397b-0b40-4efe-8c0c-654694dd3082 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:31.419747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4397b-0b40-4efe-8c0c-654694dd3082 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:31.420271 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4397b-0b40-4efe-8c0c-654694dd3082 None admin] PATH_INFO: `/v3/services` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:31.421091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4397b-0b40-4efe-8c0c-654694dd3082 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:31.421654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4397b-0b40-4efe-8c0c-654694dd3082 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:31.423328 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4397b-0b40-4efe-8c0c-654694dd3082 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:31.429247 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 366/721] 10.222.0.44 () {56 vars in 1124 bytes} [Sun Sep 11 14:59:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:31.430478 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4397b-0b40-4efe-8c0c-654694dd3082 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:31.483948 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224db38e-87bc-4879-9057-e348884150ec None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:31.530986 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c2046c-3a74-4e2f-bfad-d6ee38190aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:31.531607 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2046c-3a74-4e2f-bfad-d6ee38190aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:31.531874 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2046c-3a74-4e2f-bfad-d6ee38190aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:31.532133 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2046c-3a74-4e2f-bfad-d6ee38190aa4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:31.533594 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c2046c-3a74-4e2f-bfad-d6ee38190aa4 None admin] RBAC: Adding query filter params (role.id=c426875a15834aa5b122e2a62267d15a, scope.project.id=70ac515938a24c45ae6302dbac46a1ab, user.id=453297dac64046d19fc1c331d2d85abc) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:31.533922 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c2046c-3a74-4e2f-bfad-d6ee38190aa4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:31.535383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c2046c-3a74-4e2f-bfad-d6ee38190aa4 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:31.543640 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 356/722] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:59:31 2022] GET /identity/v3/role_assignments?user.id=453297dac64046d19fc1c331d2d85abc&scope.project.id=70ac515938a24c45ae6302dbac46a1ab&role.id=c426875a15834aa5b122e2a62267d15a => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:31.544625 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c2046c-3a74-4e2f-bfad-d6ee38190aa4 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:31.604687 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4397b-0b40-4efe-8c0c-654694dd3082 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:31.672007 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e237b-a294-4df1-b2c7-2e12f2d804f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:31.673014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:31.674120 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e237b-a294-4df1-b2c7-2e12f2d804f9 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:31.675035 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e237b-a294-4df1-b2c7-2e12f2d804f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:31.675900 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e237b-a294-4df1-b2c7-2e12f2d804f9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:31.691015 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c2046c-3a74-4e2f-bfad-d6ee38190aa4 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:31.725597 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e237b-a294-4df1-b2c7-2e12f2d804f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:59:31.736075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-f66e237b-a294-4df1-b2c7-2e12f2d804f9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:31.736075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:31.736075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:31.736075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:31.736075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:31.736075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:31.736075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:31.736075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:31.736075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:31.736075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:31.736075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:31.736075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:31.736075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:59:31.736075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:59:31.736075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:59:31.736075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:59:31.738630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:59:31.738630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:59:31.738630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:59:31.738630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:59:31.738630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:31.738630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:31.738630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:31.738630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:31.738630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:31.738630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:31.738630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:31.738630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:31.738630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:31.738630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:31.738630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:31.738630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:31.740218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:31.740218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:31.740218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:31.740218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:31.740218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:59:31.740218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:59:31.740218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:59:31.740218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:59:31.740218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:59:31.740218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:59:31.740218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:31.740218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:31.742303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fdc050-8da7-4e46-b5f3-32e5c909a95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:31.742670 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fdc050-8da7-4e46-b5f3-32e5c909a95f None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:31.742900 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fdc050-8da7-4e46-b5f3-32e5c909a95f None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:31.743125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fdc050-8da7-4e46-b5f3-32e5c909a95f None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:31.743673 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fdc050-8da7-4e46-b5f3-32e5c909a95f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:31.743976 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fdc050-8da7-4e46-b5f3-32e5c909a95f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:31.745604 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fdc050-8da7-4e46-b5f3-32e5c909a95f None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:31.747227 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 367/723] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 14:59:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:31.753930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e237b-a294-4df1-b2c7-2e12f2d804f9 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:31.754230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 357/724] 10.222.0.44 () {56 vars in 1124 bytes} [Sun Sep 11 14:59:31 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:31.767868 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:31.769284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fdc050-8da7-4e46-b5f3-32e5c909a95f None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:31.981613 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e237b-a294-4df1-b2c7-2e12f2d804f9 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:32.008466 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fdc050-8da7-4e46-b5f3-32e5c909a95f None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:32.041641 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa625bb-63f0-4703-a751-eeb628ace344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:32.042298 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa625bb-63f0-4703-a751-eeb628ace344 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:32.042648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa625bb-63f0-4703-a751-eeb628ace344 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:32.042914 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa625bb-63f0-4703-a751-eeb628ace344 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:32.055565 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa625bb-63f0-4703-a751-eeb628ace344 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:59:32.061300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-4fa625bb-63f0-4703-a751-eeb628ace344 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:59:32.061300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:32.061300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:32.061300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:32.061300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:32.061300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:32.061300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:32.061300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:32.061300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:32.061300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:32.061300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:32.061300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:32.061300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:59:32.061300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:59:32.061300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:59:32.061300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:59:32.063142 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:59:32.063142 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:59:32.063142 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:59:32.063142 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:59:32.063142 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:32.063142 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:32.063142 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:32.063142 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:32.063142 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:32.063142 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:32.063142 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:32.063142 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:32.063142 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:32.063142 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:32.063142 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:32.063142 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:32.064773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:32.064773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:32.064773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:32.064773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:32.064773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:59:32.064773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:59:32.064773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:59:32.064773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:59:32.064773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:59:32.064773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:59:32.064773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:59:32.064773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:32.064773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 368/725] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 14:59:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:32.066142 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0761c4-5c18-482f-9c9d-11c2f6695aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:32.066142 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0761c4-5c18-482f-9c9d-11c2f6695aef None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:32.066414 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0761c4-5c18-482f-9c9d-11c2f6695aef None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:32.067485 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0761c4-5c18-482f-9c9d-11c2f6695aef None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:32.067485 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0761c4-5c18-482f-9c9d-11c2f6695aef None admin] RBAC: Adding query filter params (name=service) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:32.067964 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0761c4-5c18-482f-9c9d-11c2f6695aef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:32.069730 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0761c4-5c18-482f-9c9d-11c2f6695aef None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:32.075623 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa625bb-63f0-4703-a751-eeb628ace344 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:32.079660 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 358/726] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 14:59:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:32.098953 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0761c4-5c18-482f-9c9d-11c2f6695aef None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:32.153000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:32.210063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:32.279400 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa625bb-63f0-4703-a751-eeb628ace344 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:32.312569 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0761c4-5c18-482f-9c9d-11c2f6695aef None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:32.349691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def853d6-8597-428e-92cd-a731d42612f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:32.351050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def853d6-8597-428e-92cd-a731d42612f8 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:32.351342 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e6561-8f35-4c71-a969-c21923a8531c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:32.352071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e6561-8f35-4c71-a969-c21923a8531c None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:32.352447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e6561-8f35-4c71-a969-c21923a8531c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:32.352815 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e6561-8f35-4c71-a969-c21923a8531c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:32.353162 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def853d6-8597-428e-92cd-a731d42612f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:32.353852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def853d6-8597-428e-92cd-a731d42612f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:32.355705 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def853d6-8597-428e-92cd-a731d42612f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:32.356581 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def853d6-8597-428e-92cd-a731d42612f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:32.359402 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def853d6-8597-428e-92cd-a731d42612f8 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:32.365970 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e6561-8f35-4c71-a969-c21923a8531c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:32.367924 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e6561-8f35-4c71-a969-c21923a8531c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:32.380260 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 369/727] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:59:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:59:32.383022 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2773f5-01a3-4340-85d6-2b5efa963abf None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:32.383726 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2773f5-01a3-4340-85d6-2b5efa963abf None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:32.384393 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2773f5-01a3-4340-85d6-2b5efa963abf None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:32.385524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-e74e6561-8f35-4c71-a969-c21923a8531c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:32.385524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:32.385524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:32.385524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:32.385524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:32.385524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:32.385524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:32.385524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:32.385524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:32.385524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:32.385524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:32.385524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:32.385524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:32.385524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:32.385524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:32.385524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:32.387361 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:32.387361 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:32.387361 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:32.387361 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:32.387361 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:32.387361 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:32.387361 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:32.387361 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:32.387361 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:32.387361 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:32.387361 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:32.387361 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:32.387361 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:32.387361 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:32.387361 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:32.387361 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:32.387361 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:32.389311 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:32.389311 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:32.389311 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:32.389311 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 359/728] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:32.397038 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e6561-8f35-4c71-a969-c21923a8531c None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:32.399473 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 370/729] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:32 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:32.401432 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2773f5-01a3-4340-85d6-2b5efa963abf None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:32.473268 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:32.519164 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e6561-8f35-4c71-a969-c21923a8531c None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:32.560466 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae62b171-dd8c-41ca-85ab-3cba49d8c972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:32.561207 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62b171-dd8c-41ca-85ab-3cba49d8c972 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:32.561592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62b171-dd8c-41ca-85ab-3cba49d8c972 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:32.561962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62b171-dd8c-41ca-85ab-3cba49d8c972 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:32.574865 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae62b171-dd8c-41ca-85ab-3cba49d8c972 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:32.576637 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae62b171-dd8c-41ca-85ab-3cba49d8c972 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:32.593324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-ae62b171-dd8c-41ca-85ab-3cba49d8c972 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:32.593324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:32.593324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:32.593324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:32.593324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:32.593324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:32.593324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:32.593324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:32.593324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:32.593324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:32.593324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:32.593324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:32.593324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:32.593324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:32.593324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:32.593324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:32.595259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:32.595259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:32.595259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:32.595259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:32.595259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:32.595259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:32.595259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:32.595259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:32.595259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:32.595259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:32.595259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:32.595259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:32.595259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:32.595259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:32.595259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:32.595259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:32.595259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:32.597698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:32.597698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:32.597698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:32.597698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 360/730] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:32.599326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaf69e-c01d-41b3-884b-921c02486ab6 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:32.599878 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaf69e-c01d-41b3-884b-921c02486ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:32.600107 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaf69e-c01d-41b3-884b-921c02486ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:32.621257 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-bcdaf69e-c01d-41b3-884b-921c02486ab6 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:32.623148 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-bcdaf69e-c01d-41b3-884b-921c02486ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:32.639260 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2773f5-01a3-4340-85d6-2b5efa963abf None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:32.689860 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:32.707834 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf149645-7869-4d9d-8ddf-aa3a2df6f885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:32.709787 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf149645-7869-4d9d-8ddf-aa3a2df6f885 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:32.710569 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf149645-7869-4d9d-8ddf-aa3a2df6f885 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:32.711104 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf149645-7869-4d9d-8ddf-aa3a2df6f885 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:32.712374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf149645-7869-4d9d-8ddf-aa3a2df6f885 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:32.712958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf149645-7869-4d9d-8ddf-aa3a2df6f885 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:32.717710 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf149645-7869-4d9d-8ddf-aa3a2df6f885 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:32.718378 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 361/731] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:59:32.719623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdaf69e-c01d-41b3-884b-921c02486ab6 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:32.741205 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 371/732] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:32.746857 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf149645-7869-4d9d-8ddf-aa3a2df6f885 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:32.826300 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:32.925351 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdaf69e-c01d-41b3-884b-921c02486ab6 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:32.945730 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf149645-7869-4d9d-8ddf-aa3a2df6f885 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:33.015204 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d505e-d3c9-41ee-ac3f-c738214184d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:33.016741 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d505e-d3c9-41ee-ac3f-c738214184d3 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:33.017506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d505e-d3c9-41ee-ac3f-c738214184d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:33.018306 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d505e-d3c9-41ee-ac3f-c738214184d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:33.019661 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d505e-d3c9-41ee-ac3f-c738214184d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:33.020214 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6fbd3-f33f-4c91-98c5-1b4d33cf4ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:33.020709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d505e-d3c9-41ee-ac3f-c738214184d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:33.021254 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6fbd3-f33f-4c91-98c5-1b4d33cf4ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:33.021712 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6fbd3-f33f-4c91-98c5-1b4d33cf4ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:33.022164 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6fbd3-f33f-4c91-98c5-1b4d33cf4ba2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:33.024393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d505e-d3c9-41ee-ac3f-c738214184d3 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:33.049228 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af6fbd3-f33f-4c91-98c5-1b4d33cf4ba2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:33.049228 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af6fbd3-f33f-4c91-98c5-1b4d33cf4ba2 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:33.059547 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 362/733] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:33.065225 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d505e-d3c9-41ee-ac3f-c738214184d3 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:33.071069 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-7af6fbd3-f33f-4c91-98c5-1b4d33cf4ba2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 14:59:33.071069 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:33.071069 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:33.071069 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:33.071069 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:33.071069 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:33.071069 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:33.071069 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:33.071069 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:33.071069 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:33.071069 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:33.071069 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:33.071069 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:59:33.071069 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:59:33.071069 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:59:33.071069 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:59:33.072924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:33.072924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:33.072924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:59:33.072924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:33.072924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:59:33.072924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:33.072924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:33.072924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:33.072924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:33.072924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:33.072924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:33.072924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:33.072924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:33.072924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:33.072924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:33.072924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:33.072924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:33.072924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:33.074842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:33.074842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:33.074842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:59:33.074842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:59:33.074842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:59:33.074842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:59:33.074842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:59:33.074842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:59:33.074842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 14:59:33.074842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:33.077639 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 372/734] 10.222.0.44 () {56 vars in 1117 bytes} [Sun Sep 11 14:59:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:33.082500 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6fbd3-f33f-4c91-98c5-1b4d33cf4ba2 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:33.170172 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:33.261315 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d505e-d3c9-41ee-ac3f-c738214184d3 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:33.263472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6fbd3-f33f-4c91-98c5-1b4d33cf4ba2 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:33.280419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:33.310937 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c86115b-bbf2-4ebb-8597-6f8d547b5473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:33.311977 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86115b-bbf2-4ebb-8597-6f8d547b5473 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:33.312444 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86115b-bbf2-4ebb-8597-6f8d547b5473 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:33.312900 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86115b-bbf2-4ebb-8597-6f8d547b5473 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:33.321623 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f5b15-b3a2-4581-a807-fcf3be514fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:33.322662 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f5b15-b3a2-4581-a807-fcf3be514fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:33.323098 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f5b15-b3a2-4581-a807-fcf3be514fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:33.323424 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c86115b-bbf2-4ebb-8597-6f8d547b5473 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:33.323560 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f5b15-b3a2-4581-a807-fcf3be514fd8 None admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:33.324335 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f5b15-b3a2-4581-a807-fcf3be514fd8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:33.324684 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f5b15-b3a2-4581-a807-fcf3be514fd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:33.325752 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c86115b-bbf2-4ebb-8597-6f8d547b5473 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:33.326933 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f5b15-b3a2-4581-a807-fcf3be514fd8 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:33.340094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-3c86115b-bbf2-4ebb-8597-6f8d547b5473 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 14:59:33.340094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:33.340094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:33.340094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:33.340094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:33.340094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:33.340094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:33.340094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:33.340094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:33.340094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:33.340094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:33.340094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:33.340094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:59:33.340094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:59:33.340094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:59:33.340094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:59:33.341078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:33.341078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:33.341078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:59:33.341078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:33.341078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:59:33.341078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:33.341078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:33.341078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:33.341078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:33.341078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:33.341078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:33.341078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:33.341078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:33.341078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:33.341078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:33.341078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:33.341078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:33.341078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:33.342061 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:33.342061 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:33.342061 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:59:33.342061 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:59:33.342061 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:59:33.342061 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:59:33.342061 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:59:33.342061 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:59:33.342061 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 14:59:33.342061 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:33.344895 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 363/735] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 14:59:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:33.346615 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c86115b-bbf2-4ebb-8597-6f8d547b5473 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:33.354991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 373/736] 10.222.0.44 () {56 vars in 1149 bytes} [Sun Sep 11 14:59:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:59:33.356097 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f5b15-b3a2-4581-a807-fcf3be514fd8 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:33.462473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c86115b-bbf2-4ebb-8597-6f8d547b5473 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:33.465684 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f5b15-b3a2-4581-a807-fcf3be514fd8 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:33.468330 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:33.504102 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637a601-9a92-478d-9590-b5023e39add0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:33.504823 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637a601-9a92-478d-9590-b5023e39add0 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:33.505104 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637a601-9a92-478d-9590-b5023e39add0 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:33.505405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637a601-9a92-478d-9590-b5023e39add0 None admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:33.505949 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed391d-5caa-440a-a95b-a35c994e61f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:33.506139 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637a601-9a92-478d-9590-b5023e39add0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:33.506506 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637a601-9a92-478d-9590-b5023e39add0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:33.507123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed391d-5caa-440a-a95b-a35c994e61f1 None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:33.507590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed391d-5caa-440a-a95b-a35c994e61f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:33.507935 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637a601-9a92-478d-9590-b5023e39add0 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:33.508251 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed391d-5caa-440a-a95b-a35c994e61f1 None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:33.509085 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eed391d-5caa-440a-a95b-a35c994e61f1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:33.511185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eed391d-5caa-440a-a95b-a35c994e61f1 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:33.533228 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 374/737] 10.222.0.44 () {56 vars in 1135 bytes} [Sun Sep 11 14:59:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:59:33.534382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637a601-9a92-478d-9590-b5023e39add0 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:33.543972 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 364/738] 10.222.0.44 () {60 vars in 1141 bytes} [Sun Sep 11 14:59:33 2022] POST /identity/v3/services => generated 240 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:59:33.545382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed391d-5caa-440a-a95b-a35c994e61f1 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:33.598701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:33.660829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637a601-9a92-478d-9590-b5023e39add0 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:33.718029 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c82e4-1a09-41e5-9d04-d1248b303221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:33.718930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c82e4-1a09-41e5-9d04-d1248b303221 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:33.719354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c82e4-1a09-41e5-9d04-d1248b303221 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:33.719778 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c82e4-1a09-41e5-9d04-d1248b303221 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:33.752332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:33.779343 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed391d-5caa-440a-a95b-a35c994e61f1 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:33.825170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c82e4-1a09-41e5-9d04-d1248b303221 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:33.826664 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c82e4-1a09-41e5-9d04-d1248b303221 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:33.852018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a37303-d5f6-487f-bc01-1cf1241e18be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:33.852562 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a37303-d5f6-487f-bc01-1cf1241e18be None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:33.852833 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a37303-d5f6-487f-bc01-1cf1241e18be None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:33.853098 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a37303-d5f6-487f-bc01-1cf1241e18be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:33.936593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a37303-d5f6-487f-bc01-1cf1241e18be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:33.938236 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a37303-d5f6-487f-bc01-1cf1241e18be None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:33.953059 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 375/739] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:59:33 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:59:33.959201 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c82e4-1a09-41e5-9d04-d1248b303221 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:33.989591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:34.038256 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:34.046509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 365/740] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:59:33 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 14:59:34.057113 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a37303-d5f6-487f-bc01-1cf1241e18be None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:34.143894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c82e4-1a09-41e5-9d04-d1248b303221 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:34.214176 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029ae4f-094a-450e-b828-a923ca00ad21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:34.215192 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029ae4f-094a-450e-b828-a923ca00ad21 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:34.215728 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029ae4f-094a-450e-b828-a923ca00ad21 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:34.216198 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029ae4f-094a-450e-b828-a923ca00ad21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:34.234530 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e029ae4f-094a-450e-b828-a923ca00ad21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:34.237356 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e029ae4f-094a-450e-b828-a923ca00ad21 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:34.264385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-e029ae4f-094a-450e-b828-a923ca00ad21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:34.264385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:34.264385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:34.264385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:34.264385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:34.264385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:34.264385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:34.264385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:34.264385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:34.264385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:34.264385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:34.264385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:34.264385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:34.264385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:34.264385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:34.264385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:34.265439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:34.265439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:34.265439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:34.265439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:34.265439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:34.265439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:34.265439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:34.265439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:34.265439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:34.265439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:34.265439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:34.265439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:34.265439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:34.265439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:34.265439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:34.265439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:34.265439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:34.266321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:34.266321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:34.266321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:34.271633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 376/741] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:59:34.278289 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029ae4f-094a-450e-b828-a923ca00ad21 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:34.302908 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a37303-d5f6-487f-bc01-1cf1241e18be None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:34.356562 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:34.372175 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b839595-a983-4881-bb51-6003e42fdf6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:34.374138 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b839595-a983-4881-bb51-6003e42fdf6a None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:34.375401 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b839595-a983-4881-bb51-6003e42fdf6a None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:34.375990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b839595-a983-4881-bb51-6003e42fdf6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:34.398350 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b839595-a983-4881-bb51-6003e42fdf6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:34.400383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b839595-a983-4881-bb51-6003e42fdf6a None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:34.434946 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-2b839595-a983-4881-bb51-6003e42fdf6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:34.434946 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:34.434946 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:34.434946 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:34.434946 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:34.434946 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:34.434946 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:34.434946 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:34.434946 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:34.434946 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:34.434946 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:34.434946 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:34.434946 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:34.434946 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:34.434946 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:34.434946 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:34.436226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:34.436226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:34.436226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:34.436226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:34.436226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:34.436226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:34.436226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:34.436226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:34.436226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:34.436226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:34.436226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:34.436226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:34.436226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:34.436226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:34.436226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:34.436226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:34.436226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:34.437243 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:34.437243 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:34.437243 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:34.443337 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 366/742] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:34.447686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b839595-a983-4881-bb51-6003e42fdf6a None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:34.507064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029ae4f-094a-450e-b828-a923ca00ad21 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:34.527859 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:34.563888 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8db7d-4e4d-49e1-844d-c393c2479902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:34.564516 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8db7d-4e4d-49e1-844d-c393c2479902 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:34.564782 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8db7d-4e4d-49e1-844d-c393c2479902 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:34.565042 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8db7d-4e4d-49e1-844d-c393c2479902 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:34.565714 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8db7d-4e4d-49e1-844d-c393c2479902 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:34.566007 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8db7d-4e4d-49e1-844d-c393c2479902 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:34.567376 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8db7d-4e4d-49e1-844d-c393c2479902 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:34.585174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 377/743] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:34.587107 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12765266-1b63-4539-8235-e2ca6135715e None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:34.587681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12765266-1b63-4539-8235-e2ca6135715e None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:34.588185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12765266-1b63-4539-8235-e2ca6135715e None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:34.594864 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 378/744] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:34 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:34.596441 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12765266-1b63-4539-8235-e2ca6135715e None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:34.617739 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b839595-a983-4881-bb51-6003e42fdf6a None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:34.653725 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f277e7-2248-4f52-9c12-64572ad44b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:34.654713 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f277e7-2248-4f52-9c12-64572ad44b82 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:34.655254 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f277e7-2248-4f52-9c12-64572ad44b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:34.655717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f277e7-2248-4f52-9c12-64572ad44b82 None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:34.656531 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f277e7-2248-4f52-9c12-64572ad44b82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:34.657016 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f277e7-2248-4f52-9c12-64572ad44b82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:34.658615 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f277e7-2248-4f52-9c12-64572ad44b82 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:34.672101 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 367/745] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:59:34.673710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936e1f8-3c90-470f-a508-ee692b3aec85 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:34.673990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936e1f8-3c90-470f-a508-ee692b3aec85 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:34.674261 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936e1f8-3c90-470f-a508-ee692b3aec85 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:34.675728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 368/746] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:34.676674 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3bc26-7cb8-4f1c-8645-9de69a2a1101 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:34.676946 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3bc26-7cb8-4f1c-8645-9de69a2a1101 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:34.677248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3bc26-7cb8-4f1c-8645-9de69a2a1101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:34.697370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-88a3bc26-7cb8-4f1c-8645-9de69a2a1101 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:34.698686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-88a3bc26-7cb8-4f1c-8645-9de69a2a1101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:34.724965 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12765266-1b63-4539-8235-e2ca6135715e None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:34.738966 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:34.767909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21047af1-ebde-4c5d-aa35-6f7c87fa3e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:34.768558 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21047af1-ebde-4c5d-aa35-6f7c87fa3e04 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:34.768835 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21047af1-ebde-4c5d-aa35-6f7c87fa3e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:34.769108 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21047af1-ebde-4c5d-aa35-6f7c87fa3e04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:34.778177 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21047af1-ebde-4c5d-aa35-6f7c87fa3e04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:34.779429 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 369/747] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:34 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:59:34.780002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21047af1-ebde-4c5d-aa35-6f7c87fa3e04 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:34.780272 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3bc26-7cb8-4f1c-8645-9de69a2a1101 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:34.802965 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-21047af1-ebde-4c5d-aa35-6f7c87fa3e04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:34.802965 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:34.802965 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:34.802965 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:34.802965 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:34.802965 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:34.802965 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:34.802965 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:34.802965 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:34.802965 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:34.802965 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:34.802965 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:34.802965 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:59:34.802965 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:59:34.802965 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:59:34.804080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:59:34.804080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:34.804080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:34.804080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:34.804080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:34.804080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:34.804080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:34.804080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:34.804080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:34.804080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:34.804080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:34.804080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:34.804080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:34.804080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:34.804080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:34.804080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:34.804080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:34.805361 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:59:34.805361 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:59:34.805361 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:59:34.805361 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:59:34.805361 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:59:34.805361 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:59:34.805361 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:34.805361 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:34.808487 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 379/748] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:59:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:34.810113 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e141f88-ff34-471d-8b5b-3036101276c1 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:34.810521 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e141f88-ff34-471d-8b5b-3036101276c1 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:34.810910 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e141f88-ff34-471d-8b5b-3036101276c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:34.832407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-2e141f88-ff34-471d-8b5b-3036101276c1 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:34.833993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-2e141f88-ff34-471d-8b5b-3036101276c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:34.865529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:34.894667 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3bc26-7cb8-4f1c-8645-9de69a2a1101 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:34.927527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 380/749] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:34 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 14:59:34.928538 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e141f88-ff34-471d-8b5b-3036101276c1 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:34.956704 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb4861-4065-4347-8f65-91ab4704abe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:34.957382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb4861-4065-4347-8f65-91ab4704abe0 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:34.957655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb4861-4065-4347-8f65-91ab4704abe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:34.957922 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb4861-4065-4347-8f65-91ab4704abe0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:34.971712 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb4861-4065-4347-8f65-91ab4704abe0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:34.973203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb4861-4065-4347-8f65-91ab4704abe0 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:34.995366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-64eb4861-4065-4347-8f65-91ab4704abe0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:34.995366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:34.995366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:34.995366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:34.995366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:34.995366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:34.995366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:34.995366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:34.995366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:34.995366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:34.995366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:34.995366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:34.995366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:59:34.995366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:59:34.995366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:59:34.996333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:59:34.996333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:34.996333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:34.996333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:34.996333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:34.996333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:34.996333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:34.996333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:34.996333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:34.996333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:34.996333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:34.996333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:34.996333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:34.996333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:34.996333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:34.996333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:34.996333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:34.997693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:59:34.997693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:59:34.997693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:59:34.997693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:59:34.997693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:59:34.997693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:59:34.997693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:34.997693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:34.997693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 370/750] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:59:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:35.006877 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb4861-4065-4347-8f65-91ab4704abe0 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:35.067427 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:35.108824 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e141f88-ff34-471d-8b5b-3036101276c1 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:35.148971 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae923f-5551-4d64-8896-58ba28291f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:35.149629 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae923f-5551-4d64-8896-58ba28291f33 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:35.149982 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae923f-5551-4d64-8896-58ba28291f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:35.150249 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae923f-5551-4d64-8896-58ba28291f33 None admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:35.150957 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae923f-5551-4d64-8896-58ba28291f33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:35.151253 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae923f-5551-4d64-8896-58ba28291f33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:35.152214 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb4861-4065-4347-8f65-91ab4704abe0 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:35.152778 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae923f-5551-4d64-8896-58ba28291f33 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:35.165826 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 381/751] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 14:59:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:35.166783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae923f-5551-4d64-8896-58ba28291f33 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:35.188878 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b070c1b-92ee-4d77-8a2f-1986e9cb9830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:35.189894 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b070c1b-92ee-4d77-8a2f-1986e9cb9830 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:35.190383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b070c1b-92ee-4d77-8a2f-1986e9cb9830 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:35.190852 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b070c1b-92ee-4d77-8a2f-1986e9cb9830 None admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:35.191745 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b070c1b-92ee-4d77-8a2f-1986e9cb9830 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:35.192254 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b070c1b-92ee-4d77-8a2f-1986e9cb9830 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:35.193978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b070c1b-92ee-4d77-8a2f-1986e9cb9830 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:35.197861 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:35.208744 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 371/752] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 14:59:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:35.210041 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b070c1b-92ee-4d77-8a2f-1986e9cb9830 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:35.307369 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae923f-5551-4d64-8896-58ba28291f33 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:35.319750 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:35.331299 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b070c1b-92ee-4d77-8a2f-1986e9cb9830 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:35.344861 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e6d55-2197-4ebc-a3f8-73de84f0259f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:35.345532 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e6d55-2197-4ebc-a3f8-73de84f0259f None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:35.345805 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e6d55-2197-4ebc-a3f8-73de84f0259f None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:35.346427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e6d55-2197-4ebc-a3f8-73de84f0259f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:35.357519 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9e6d55-2197-4ebc-a3f8-73de84f0259f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:59:35.362846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-0c9e6d55-2197-4ebc-a3f8-73de84f0259f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:59:35.362846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:35.362846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:35.362846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:35.362846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:35.362846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:35.362846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:35.362846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:35.362846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:35.362846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:35.362846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:35.362846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:35.362846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:59:35.362846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:59:35.362846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:59:35.362846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:59:35.365203 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:59:35.365203 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:59:35.365203 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:59:35.365203 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:59:35.365203 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:35.365203 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:35.365203 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:35.365203 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:35.365203 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:35.365203 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:35.365203 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:35.365203 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:35.365203 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:35.365203 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:35.365203 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:35.365203 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:35.366138 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:35.366138 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:35.366138 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:35.366138 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:35.366138 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:59:35.366138 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:59:35.366138 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:59:35.366138 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:59:35.366138 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:59:35.366138 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:59:35.366138 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:59:35.366138 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:35.366138 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 382/753] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 14:59:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:35.366138 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e6d55-2197-4ebc-a3f8-73de84f0259f None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:35.375015 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955de17a-b05b-4878-a305-977732c90171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:35.375800 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955de17a-b05b-4878-a305-977732c90171 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:35.376167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955de17a-b05b-4878-a305-977732c90171 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:35.376536 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955de17a-b05b-4878-a305-977732c90171 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:35.377397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955de17a-b05b-4878-a305-977732c90171 None admin] RBAC: Adding query filter params (role.id=ea3b2f1eb283418b8823cb4fff80edba, scope.project.id=70ac515938a24c45ae6302dbac46a1ab, user.id=3fada78ceaeb4f3bad77c516f6dfcfd4) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:35.377793 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955de17a-b05b-4878-a305-977732c90171 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:35.379301 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955de17a-b05b-4878-a305-977732c90171 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:35.385609 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 372/754] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:59:35 2022] GET /identity/v3/role_assignments?user.id=3fada78ceaeb4f3bad77c516f6dfcfd4&scope.project.id=70ac515938a24c45ae6302dbac46a1ab&role.id=ea3b2f1eb283418b8823cb4fff80edba => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:35.386363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955de17a-b05b-4878-a305-977732c90171 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:35.491951 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:35.516329 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955de17a-b05b-4878-a305-977732c90171 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:35.517952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e6d55-2197-4ebc-a3f8-73de84f0259f None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:35.562164 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36977646-d3ba-4973-be88-68f7283d841f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:35.562947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36977646-d3ba-4973-be88-68f7283d841f None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:35.563291 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36977646-d3ba-4973-be88-68f7283d841f None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:35.563652 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36977646-d3ba-4973-be88-68f7283d841f None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:35.564420 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36977646-d3ba-4973-be88-68f7283d841f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:35.566012 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36977646-d3ba-4973-be88-68f7283d841f None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:35.568361 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86351e-ae05-4288-95ba-d1cfc5a12874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:35.569335 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86351e-ae05-4288-95ba-d1cfc5a12874 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:35.569790 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86351e-ae05-4288-95ba-d1cfc5a12874 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:35.570498 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86351e-ae05-4288-95ba-d1cfc5a12874 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:35.572081 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86351e-ae05-4288-95ba-d1cfc5a12874 None admin] RBAC: Adding query filter params (role.id=c426875a15834aa5b122e2a62267d15a, scope.project.id=70ac515938a24c45ae6302dbac46a1ab, user.id=05aa5e40ee4542719657bc03b2015e72) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:35.572615 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86351e-ae05-4288-95ba-d1cfc5a12874 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:35.574495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86351e-ae05-4288-95ba-d1cfc5a12874 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:35.582762 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 383/755] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:59:35 2022] GET /identity/v3/role_assignments?user.id=05aa5e40ee4542719657bc03b2015e72&scope.project.id=70ac515938a24c45ae6302dbac46a1ab&role.id=c426875a15834aa5b122e2a62267d15a => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:35.583890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86351e-ae05-4288-95ba-d1cfc5a12874 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:35.586516 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 373/756] 10.222.0.44 () {60 vars in 1141 bytes} [Sun Sep 11 14:59:35 2022] POST /identity/v3/services => generated 251 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:59:35.720033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86351e-ae05-4288-95ba-d1cfc5a12874 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:35.770057 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedc228-fe9d-46d8-b459-441cdcff4602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:35.770719 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedc228-fe9d-46d8-b459-441cdcff4602 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:35.770955 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedc228-fe9d-46d8-b459-441cdcff4602 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:35.771233 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedc228-fe9d-46d8-b459-441cdcff4602 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:35.771900 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecedc228-fe9d-46d8-b459-441cdcff4602 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:35.772202 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecedc228-fe9d-46d8-b459-441cdcff4602 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:35.773698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecedc228-fe9d-46d8-b459-441cdcff4602 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:35.785468 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 384/757] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:59:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:35.795446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36977646-d3ba-4973-be88-68f7283d841f None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:35.846861 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:35.964285 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36977646-d3ba-4973-be88-68f7283d841f None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:36.006288 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bfb693-5466-4122-a707-c3f53508f3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:36.007002 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bfb693-5466-4122-a707-c3f53508f3ce None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:36.007289 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bfb693-5466-4122-a707-c3f53508f3ce None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:36.007608 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bfb693-5466-4122-a707-c3f53508f3ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:36.020867 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bfb693-5466-4122-a707-c3f53508f3ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:36.022334 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bfb693-5466-4122-a707-c3f53508f3ce None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:36.039507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-01bfb693-5466-4122-a707-c3f53508f3ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:36.039507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:36.039507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:36.039507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:36.039507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:36.039507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:36.039507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:36.039507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:36.039507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:36.039507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:36.039507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:36.039507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:36.039507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:36.039507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:36.039507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:36.039507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:36.040524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:36.040524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:36.040524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:36.040524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:36.040524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:36.040524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:36.040524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:36.040524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:36.040524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:36.040524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:36.040524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:36.040524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:36.040524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:36.040524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:36.040524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:36.040524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:36.040524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:36.041488 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:36.041488 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:36.041488 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:36.041488 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 374/758] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:36.050608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedc228-fe9d-46d8-b459-441cdcff4602 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:36.053786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:36.174339 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedc228-fe9d-46d8-b459-441cdcff4602 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:36.220532 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ba593-7383-4673-b6e2-82a6d59b6ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:36.221570 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ba593-7383-4673-b6e2-82a6d59b6ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:36.221847 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ba593-7383-4673-b6e2-82a6d59b6ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:36.222114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ba593-7383-4673-b6e2-82a6d59b6ad4 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:36.222788 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ba593-7383-4673-b6e2-82a6d59b6ad4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:36.223085 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ba593-7383-4673-b6e2-82a6d59b6ad4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:36.224498 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ba593-7383-4673-b6e2-82a6d59b6ad4 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:36.240464 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 385/759] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:36.246659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bfb693-5466-4122-a707-c3f53508f3ce None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:36.377470 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:36.433113 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bfb693-5466-4122-a707-c3f53508f3ce None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:36.491277 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b98ce6-5fce-49cb-a62f-e4edbe5707fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:36.491942 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b98ce6-5fce-49cb-a62f-e4edbe5707fc None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:36.492219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b98ce6-5fce-49cb-a62f-e4edbe5707fc None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:36.492482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b98ce6-5fce-49cb-a62f-e4edbe5707fc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:36.509737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b98ce6-5fce-49cb-a62f-e4edbe5707fc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:36.511384 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b98ce6-5fce-49cb-a62f-e4edbe5707fc None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:36.528813 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-44b98ce6-5fce-49cb-a62f-e4edbe5707fc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:59:36.528813 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:36.528813 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:36.528813 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:36.528813 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:36.528813 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:36.528813 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:36.528813 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:36.528813 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:36.528813 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:36.528813 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:36.528813 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:36.528813 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:59:36.528813 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:59:36.528813 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:59:36.528813 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:59:36.531044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:36.531044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:36.531044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:59:36.531044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:36.531044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:59:36.531044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:36.531044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:36.531044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:36.531044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:36.531044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:36.531044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:36.531044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:36.531044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:36.531044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:36.531044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:36.531044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:36.531044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:36.531044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:36.533318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:36.533318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:36.533318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:59:36.533318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:59:36.533318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:59:36.533318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:59:36.533318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:59:36.533318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:59:36.533318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:59:36.533318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:36.533318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 375/760] 10.222.0.44 () {56 vars in 1093 bytes} [Sun Sep 11 14:59:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:36.537018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ba593-7383-4673-b6e2-82a6d59b6ad4 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:36.648144 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:36.704943 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ba593-7383-4673-b6e2-82a6d59b6ad4 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:36.748694 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca57684-9251-4cb2-8ae1-cc6919f85389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:36.749356 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca57684-9251-4cb2-8ae1-cc6919f85389 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:36.749637 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca57684-9251-4cb2-8ae1-cc6919f85389 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:36.749880 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca57684-9251-4cb2-8ae1-cc6919f85389 None admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:36.750612 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca57684-9251-4cb2-8ae1-cc6919f85389 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:36.750919 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca57684-9251-4cb2-8ae1-cc6919f85389 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:36.752371 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca57684-9251-4cb2-8ae1-cc6919f85389 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:36.780807 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 386/761] 10.222.0.44 () {56 vars in 1133 bytes} [Sun Sep 11 14:59:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:36.786454 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b98ce6-5fce-49cb-a62f-e4edbe5707fc None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:36.942439 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b98ce6-5fce-49cb-a62f-e4edbe5707fc None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:36.969727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a768ee-d6a1-4d80-9ca0-3c225ea7f622 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:36.970023 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a768ee-d6a1-4d80-9ca0-3c225ea7f622 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:36.970383 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a768ee-d6a1-4d80-9ca0-3c225ea7f622 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:36.972276 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 387/762] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:59:36.977995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc5a26-1c10-4e93-9fb0-acae03864cb5 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:36.978276 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc5a26-1c10-4e93-9fb0-acae03864cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:36.978582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc5a26-1c10-4e93-9fb0-acae03864cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:36.978742 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:37.002409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8148f-ca9f-43d3-9839-759198f5c077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:37.003070 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8148f-ca9f-43d3-9839-759198f5c077 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:37.003347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8148f-ca9f-43d3-9839-759198f5c077 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:37.003606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8148f-ca9f-43d3-9839-759198f5c077 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:37.004478 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-cfdc5a26-1c10-4e93-9fb0-acae03864cb5 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:37.005772 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-cfdc5a26-1c10-4e93-9fb0-acae03864cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:37.031028 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8148f-ca9f-43d3-9839-759198f5c077 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:37.032560 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8148f-ca9f-43d3-9839-759198f5c077 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:37.065947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-5cc8148f-ca9f-43d3-9839-759198f5c077 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:37.065947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:37.065947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:37.065947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:37.065947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:37.065947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:37.065947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:37.065947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:37.065947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:37.065947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:37.065947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:37.065947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:37.065947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:37.065947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:37.065947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:37.065947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:37.067791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:37.067791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:37.067791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:37.067791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:37.067791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:37.067791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:37.067791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:37.067791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:37.067791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:37.067791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:37.067791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:37.067791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:37.067791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:37.067791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:37.067791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:37.067791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:37.067791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:37.069563 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:37.069563 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:37.069563 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:37.069563 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 376/763] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:37.073428 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8148f-ca9f-43d3-9839-759198f5c077 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:37.150745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 388/764] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 14:59:37.252642 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8148f-ca9f-43d3-9839-759198f5c077 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:37.307427 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b71c21-7e73-4edc-b319-acb135942c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:37.308052 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b71c21-7e73-4edc-b319-acb135942c31 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:37.308325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b71c21-7e73-4edc-b319-acb135942c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:37.308596 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b71c21-7e73-4edc-b319-acb135942c31 None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:37.309321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b71c21-7e73-4edc-b319-acb135942c31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:37.309626 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b71c21-7e73-4edc-b319-acb135942c31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:37.311039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b71c21-7e73-4edc-b319-acb135942c31 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:37.329089 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 377/765] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:37.334816 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc5a26-1c10-4e93-9fb0-acae03864cb5 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:37.391094 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:37.446999 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc5a26-1c10-4e93-9fb0-acae03864cb5 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:37.490389 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b71c21-7e73-4edc-b319-acb135942c31 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:37.490974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb8d7f-7d5b-41ec-bfff-8511b6163251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:37.491648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb8d7f-7d5b-41ec-bfff-8511b6163251 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:37.491901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb8d7f-7d5b-41ec-bfff-8511b6163251 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:37.492155 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb8d7f-7d5b-41ec-bfff-8511b6163251 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:37.501658 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb8d7f-7d5b-41ec-bfff-8511b6163251 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:37.503920 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb8d7f-7d5b-41ec-bfff-8511b6163251 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:37.531456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-43fb8d7f-7d5b-41ec-bfff-8511b6163251 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:37.531456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:37.531456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:37.531456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:37.531456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:37.531456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:37.531456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:37.531456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:37.531456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:37.531456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:37.531456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:37.531456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:37.531456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:59:37.531456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:59:37.531456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:59:37.534894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:59:37.534894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:37.534894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:37.534894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:37.534894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:37.534894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:37.534894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:37.534894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:37.534894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:37.534894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:37.534894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:37.534894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:37.534894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:37.534894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:37.534894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:37.534894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:37.534894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:37.538282 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:59:37.538282 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:59:37.538282 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:59:37.538282 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:59:37.538282 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:59:37.538282 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:59:37.538282 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:37.538282 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:37.538282 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 389/766] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:59:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:37.540277 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb8d7f-7d5b-41ec-bfff-8511b6163251 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:37.576232 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:37.705943 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb8d7f-7d5b-41ec-bfff-8511b6163251 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:37.723937 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b71c21-7e73-4edc-b319-acb135942c31 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:37.730113 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:37.783466 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9112b54-78f9-4c99-9f76-468e56cdd343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:37.786307 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9112b54-78f9-4c99-9f76-468e56cdd343 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:37.786965 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9112b54-78f9-4c99-9f76-468e56cdd343 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:37.787640 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9112b54-78f9-4c99-9f76-468e56cdd343 None admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:37.789330 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9112b54-78f9-4c99-9f76-468e56cdd343 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:37.790115 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9112b54-78f9-4c99-9f76-468e56cdd343 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:37.793019 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9112b54-78f9-4c99-9f76-468e56cdd343 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:37.810410 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd9407-d70a-4ad1-a769-52c64e1d45e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:37.811420 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd9407-d70a-4ad1-a769-52c64e1d45e8 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:37.811908 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd9407-d70a-4ad1-a769-52c64e1d45e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:37.812441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd9407-d70a-4ad1-a769-52c64e1d45e8 None admin] PATH_INFO: `/v3/services/network` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:37.813532 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd9407-d70a-4ad1-a769-52c64e1d45e8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:37.815226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd9407-d70a-4ad1-a769-52c64e1d45e8 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:37.825391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 390/767] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 14:59:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:59:37.831831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9112b54-78f9-4c99-9f76-468e56cdd343 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:37.833573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-30fd9407-d70a-4ad1-a769-52c64e1d45e8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 14:59:37.833573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:37.833573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:59:37.833573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:59:37.833573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:59:37.833573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:59:37.833573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:59:37.833573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:59:37.833573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 14:59:37.833573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:37.833573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:59:37.833573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:37.833573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:37.833573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:37.833573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:37.833573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:37.833573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:37.834599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:37.834599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:37.834599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:37.834599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:37.834599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:37.834599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:37.834599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:59:37.834599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:59:37.834599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:59:37.834599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:59:37.834599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:37.834599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:37.834599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:37.834599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:37.834599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:37.834599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:37.835449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:37.835449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:37.835449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:37.835449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:37.835449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:37.835449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:37.835449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:37.835449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:37.835449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:37.835449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:37.835449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:59:37.835449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:59:37.835449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 14:59:37.835449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:37.850602 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 378/768] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:59:37 2022] GET /identity/v3/services/network => generated 88 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:37.853066 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd9407-d70a-4ad1-a769-52c64e1d45e8 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:37.977843 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd9407-d70a-4ad1-a769-52c64e1d45e8 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:37.979769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:38.036306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9112b54-78f9-4c99-9f76-468e56cdd343 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:38.077435 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64619203-4194-42e9-8729-a6bb216dc077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:38.077700 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea23de8-559a-40ae-9e00-15de8a51bcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:38.078107 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea23de8-559a-40ae-9e00-15de8a51bcc8 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:38.078385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea23de8-559a-40ae-9e00-15de8a51bcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:38.078654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea23de8-559a-40ae-9e00-15de8a51bcc8 None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:38.079170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64619203-4194-42e9-8729-a6bb216dc077 None admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:38.079281 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea23de8-559a-40ae-9e00-15de8a51bcc8 None admin] RBAC: Adding query filter params (name=network) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:38.079584 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea23de8-559a-40ae-9e00-15de8a51bcc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:38.079900 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64619203-4194-42e9-8729-a6bb216dc077 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:38.080336 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64619203-4194-42e9-8729-a6bb216dc077 None admin] PATH_INFO: `/v3/projects/70ac515938a24c45ae6302dbac46a1ab/users/453297dac64046d19fc1c331d2d85abc/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:38.080985 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea23de8-559a-40ae-9e00-15de8a51bcc8 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:38.082884 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64619203-4194-42e9-8729-a6bb216dc077 None admin] RBAC: Authorizing `identity:create_grant(project_id=70ac515938a24c45ae6302dbac46a1ab, user_id=453297dac64046d19fc1c331d2d85abc, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:38.084551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64619203-4194-42e9-8729-a6bb216dc077 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:38.094441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 379/769] 10.222.0.44 () {56 vars in 1112 bytes} [Sun Sep 11 14:59:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:38.102321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea23de8-559a-40ae-9e00-15de8a51bcc8 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:38.106713 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 391/770] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 14:59:37 2022] PUT /identity/v3/projects/70ac515938a24c45ae6302dbac46a1ab/users/453297dac64046d19fc1c331d2d85abc/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 14:59:38.222345 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea23de8-559a-40ae-9e00-15de8a51bcc8 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:38.280789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c189a1-e389-4b52-95c0-ef200cc0cfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:38.281464 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c189a1-e389-4b52-95c0-ef200cc0cfe6 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:38.281742 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c189a1-e389-4b52-95c0-ef200cc0cfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:38.282008 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c189a1-e389-4b52-95c0-ef200cc0cfe6 None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:38.282660 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c189a1-e389-4b52-95c0-ef200cc0cfe6 None admin] RBAC: Adding query filter params (type=network) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:38.282960 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c189a1-e389-4b52-95c0-ef200cc0cfe6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:38.284412 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c189a1-e389-4b52-95c0-ef200cc0cfe6 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:38.290969 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 380/771] 10.222.0.44 () {56 vars in 1112 bytes} [Sun Sep 11 14:59:38 2022] GET /identity/v3/services?type=network => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:38.301025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64619203-4194-42e9-8729-a6bb216dc077 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:38.355107 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:38.438809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64619203-4194-42e9-8729-a6bb216dc077 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:38.478456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847a4d0-2aaf-44b2-a174-c74bd227422f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:38.479513 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847a4d0-2aaf-44b2-a174-c74bd227422f None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:38.480044 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847a4d0-2aaf-44b2-a174-c74bd227422f None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:38.480519 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847a4d0-2aaf-44b2-a174-c74bd227422f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:38.481500 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847a4d0-2aaf-44b2-a174-c74bd227422f None admin] RBAC: Adding query filter params (interface=public, service_id=6d93f334c7fd43579087143f4f1c87f8, region_id=RegionOne) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:38.482029 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847a4d0-2aaf-44b2-a174-c74bd227422f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:38.483657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847a4d0-2aaf-44b2-a174-c74bd227422f None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:38.492921 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 392/772] 10.222.0.44 () {56 vars in 1251 bytes} [Sun Sep 11 14:59:38 2022] GET /identity/v3/endpoints?service_id=6d93f334c7fd43579087143f4f1c87f8&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:38.498661 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c189a1-e389-4b52-95c0-ef200cc0cfe6 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:38.515548 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692ad84-01ca-4970-94e2-8defaecf9a1c None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:38.516254 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692ad84-01ca-4970-94e2-8defaecf9a1c None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:38.516725 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692ad84-01ca-4970-94e2-8defaecf9a1c None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:38.518615 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 393/773] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:38.532344 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4174d48-4435-4c9b-8f1f-845e2df1c138 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:38.533031 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4174d48-4435-4c9b-8f1f-845e2df1c138 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:38.533566 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4174d48-4435-4c9b-8f1f-845e2df1c138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:38.553482 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-a4174d48-4435-4c9b-8f1f-845e2df1c138 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:38.555086 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-a4174d48-4435-4c9b-8f1f-845e2df1c138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:38.561170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:38.644822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 394/774] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 14:59:38.698854 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c189a1-e389-4b52-95c0-ef200cc0cfe6 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:38.749075 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f247f6b3-ead2-4585-a085-990b53d6466a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:38.750152 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247f6b3-ead2-4585-a085-990b53d6466a None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:38.750620 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247f6b3-ead2-4585-a085-990b53d6466a None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:38.751073 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247f6b3-ead2-4585-a085-990b53d6466a None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:38.751899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f247f6b3-ead2-4585-a085-990b53d6466a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:38.753542 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f247f6b3-ead2-4585-a085-990b53d6466a None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:38.765469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 381/775] 10.222.0.44 () {56 vars in 1087 bytes} [Sun Sep 11 14:59:38 2022] GET /identity/v3/services => generated 772 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:38.843754 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd06b78-bef8-401c-bdb9-14ba5f7973ae None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:38.844767 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd06b78-bef8-401c-bdb9-14ba5f7973ae None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:38.845278 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd06b78-bef8-401c-bdb9-14ba5f7973ae None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:38.847509 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 395/776] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:38.855737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99fcd2b-4663-44bd-9ad2-d08e34543352 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:38.856384 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99fcd2b-4663-44bd-9ad2-d08e34543352 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:38.856849 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99fcd2b-4663-44bd-9ad2-d08e34543352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:38.886484 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-b99fcd2b-4663-44bd-9ad2-d08e34543352 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:38.888093 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b99fcd2b-4663-44bd-9ad2-d08e34543352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:38.978213 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 382/777] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:59:39.145737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd06b78-bef8-401c-bdb9-14ba5f7973ae None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:39.175501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:39.264287 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643d318-36eb-4525-a204-b51ad80adf29 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:39.264990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643d318-36eb-4525-a204-b51ad80adf29 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:39.265633 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643d318-36eb-4525-a204-b51ad80adf29 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:39.267477 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 383/778] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:39.273726 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b1572-805c-490f-9d09-85fedd1e98ff None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:39.274355 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b1572-805c-490f-9d09-85fedd1e98ff None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:39.274843 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b1572-805c-490f-9d09-85fedd1e98ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:39.276647 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd06b78-bef8-401c-bdb9-14ba5f7973ae None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:39.314995 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-792b1572-805c-490f-9d09-85fedd1e98ff None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:39.316448 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-792b1572-805c-490f-9d09-85fedd1e98ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:39.327328 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206e01f-ebdb-43e5-a7d5-80166bc51397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:39.327998 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206e01f-ebdb-43e5-a7d5-80166bc51397 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:39.328273 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206e01f-ebdb-43e5-a7d5-80166bc51397 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:39.328607 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206e01f-ebdb-43e5-a7d5-80166bc51397 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:39.340371 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:39.350729 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8206e01f-ebdb-43e5-a7d5-80166bc51397 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:59:39.355886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-8206e01f-ebdb-43e5-a7d5-80166bc51397 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:39.355886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:39.355886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:39.355886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:39.355886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:39.355886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:39.355886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:39.355886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:39.355886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:39.355886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:39.355886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:39.355886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:39.355886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:59:39.355886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:59:39.355886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:59:39.355886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:59:39.356896 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:59:39.356896 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:59:39.356896 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:59:39.356896 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:59:39.356896 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:39.356896 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:39.356896 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:39.356896 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:39.356896 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:39.356896 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:39.356896 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:39.356896 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:39.356896 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:39.356896 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:39.356896 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:39.356896 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:39.358239 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:39.358239 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:39.358239 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:39.358239 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:39.358239 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:59:39.358239 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:59:39.358239 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:59:39.358239 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:59:39.358239 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:59:39.358239 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:59:39.358239 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:39.358239 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:39.358239 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 396/779] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 14:59:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:39.366158 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206e01f-ebdb-43e5-a7d5-80166bc51397 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:39.417661 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 384/780] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 14:59:39.418554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b1572-805c-490f-9d09-85fedd1e98ff None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:39.498441 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:39.527929 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206e01f-ebdb-43e5-a7d5-80166bc51397 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:39.597816 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a4d85-b733-4a9b-bd40-01aad84abb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:39.598556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a4d85-b733-4a9b-bd40-01aad84abb45 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:39.598834 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a4d85-b733-4a9b-bd40-01aad84abb45 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:39.599102 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a4d85-b733-4a9b-bd40-01aad84abb45 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:39.599785 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a4d85-b733-4a9b-bd40-01aad84abb45 None admin] RBAC: Adding query filter params (name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:39.600083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a4d85-b733-4a9b-bd40-01aad84abb45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:39.601504 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a4d85-b733-4a9b-bd40-01aad84abb45 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:39.612367 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 397/781] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 14:59:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:59:39.620745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a4d85-b733-4a9b-bd40-01aad84abb45 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:39.646103 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b1572-805c-490f-9d09-85fedd1e98ff None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:39.726615 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aff92e-8ae4-4fe7-bf1e-39fe1852778f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:39.729391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aff92e-8ae4-4fe7-bf1e-39fe1852778f None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:39.730539 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aff92e-8ae4-4fe7-bf1e-39fe1852778f None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:39.734844 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aff92e-8ae4-4fe7-bf1e-39fe1852778f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:39.736360 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aff92e-8ae4-4fe7-bf1e-39fe1852778f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:39.739516 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aff92e-8ae4-4fe7-bf1e-39fe1852778f None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:39.754999 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-83aff92e-8ae4-4fe7-bf1e-39fe1852778f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 14:59:39.754999 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:39.754999 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:59:39.754999 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:59:39.754999 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:59:39.754999 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:59:39.754999 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:59:39.754999 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:59:39.754999 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 14:59:39.754999 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:39.754999 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:59:39.754999 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:39.754999 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:39.754999 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:39.754999 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:39.754999 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:39.754999 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:39.757020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:39.757020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:39.757020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:39.757020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:39.757020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:39.757020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:39.757020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:59:39.757020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:59:39.757020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:59:39.757020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:59:39.757020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:39.757020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:39.757020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:39.757020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:39.757020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:39.757020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:39.758810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:39.758810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:39.758810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:39.758810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:39.758810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:39.758810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:39.758810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:39.758810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:39.758810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:39.758810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:39.758810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:59:39.758810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:59:39.758810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 14:59:39.758810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:39.758810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 385/782] 10.222.0.44 () {56 vars in 1129 bytes} [Sun Sep 11 14:59:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:39.762576 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aff92e-8ae4-4fe7-bf1e-39fe1852778f None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:39.807699 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:39.893698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a4d85-b733-4a9b-bd40-01aad84abb45 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:39.955043 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27253b4-1f09-4cb8-9117-f4af491a2900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:39.956086 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27253b4-1f09-4cb8-9117-f4af491a2900 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:39.956523 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27253b4-1f09-4cb8-9117-f4af491a2900 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:39.956972 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27253b4-1f09-4cb8-9117-f4af491a2900 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:39.960269 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aff92e-8ae4-4fe7-bf1e-39fe1852778f None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:39.977468 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27253b4-1f09-4cb8-9117-f4af491a2900 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:39.980210 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27253b4-1f09-4cb8-9117-f4af491a2900 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:39.994286 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:40.008607 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33f284-2c1e-4f2c-9dbb-eb86ce0232a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:40.010035 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33f284-2c1e-4f2c-9dbb-eb86ce0232a9 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:40.010524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33f284-2c1e-4f2c-9dbb-eb86ce0232a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:40.011094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33f284-2c1e-4f2c-9dbb-eb86ce0232a9 None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:40.011556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-b27253b4-1f09-4cb8-9117-f4af491a2900 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:40.011556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:40.011556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:40.011556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:40.011556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:40.011556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:40.011556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:40.011556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:40.011556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:40.011556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:40.011556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:40.011556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:40.011556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:40.011556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:40.011556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:40.011556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:40.012459 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:40.012459 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:40.012459 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:40.012459 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:40.012459 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:40.012459 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:40.012459 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:40.012459 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:40.012459 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:40.012459 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:40.012459 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:40.012459 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:40.012459 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:40.012459 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:40.012459 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:40.012459 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:40.012459 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:40.013374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:40.013374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:40.013374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:40.014035 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 398/783] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:40.014725 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb33f284-2c1e-4f2c-9dbb-eb86ce0232a9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:40.015369 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb33f284-2c1e-4f2c-9dbb-eb86ce0232a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:40.015561 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27253b4-1f09-4cb8-9117-f4af491a2900 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:40.017209 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb33f284-2c1e-4f2c-9dbb-eb86ce0232a9 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:40.023453 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 386/784] 10.222.0.44 () {56 vars in 1124 bytes} [Sun Sep 11 14:59:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:59:40.024789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33f284-2c1e-4f2c-9dbb-eb86ce0232a9 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:40.132566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:40.158459 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33f284-2c1e-4f2c-9dbb-eb86ce0232a9 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:40.202303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae3d3e-bb1a-4d53-81b8-f0916de2aaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:40.203381 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae3d3e-bb1a-4d53-81b8-f0916de2aaa7 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:40.203875 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae3d3e-bb1a-4d53-81b8-f0916de2aaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:40.204344 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae3d3e-bb1a-4d53-81b8-f0916de2aaa7 None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:40.205250 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ae3d3e-bb1a-4d53-81b8-f0916de2aaa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:40.205825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ae3d3e-bb1a-4d53-81b8-f0916de2aaa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:40.207758 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ae3d3e-bb1a-4d53-81b8-f0916de2aaa7 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:40.223495 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 387/785] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:40.224805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae3d3e-bb1a-4d53-81b8-f0916de2aaa7 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:40.274072 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27253b4-1f09-4cb8-9117-f4af491a2900 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:40.343331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cd533d-7f7d-4100-8b5e-5e868343b1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:40.344377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd533d-7f7d-4100-8b5e-5e868343b1dc None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:40.344855 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd533d-7f7d-4100-8b5e-5e868343b1dc None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:40.345352 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd533d-7f7d-4100-8b5e-5e868343b1dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:40.359324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cd533d-7f7d-4100-8b5e-5e868343b1dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:59:40.365099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-90cd533d-7f7d-4100-8b5e-5e868343b1dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:59:40.365099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:40.365099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:40.365099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:40.365099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:40.365099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:40.365099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:40.365099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:40.365099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:40.365099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:40.365099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:40.365099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:40.365099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:59:40.365099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:59:40.365099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:59:40.365099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:59:40.366094 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:59:40.366094 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:59:40.366094 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:59:40.366094 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:59:40.366094 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:40.366094 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:40.366094 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:40.366094 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:40.366094 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:40.366094 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:40.366094 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:40.366094 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:40.366094 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:40.366094 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:40.366094 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:40.366094 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:40.366930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:40.366930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:40.366930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:40.366930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:40.366930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:59:40.366930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:59:40.366930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:59:40.366930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:59:40.366930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:59:40.366930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:59:40.366930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:59:40.366930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:40.374564 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 399/786] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 14:59:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:40.375903 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cd533d-7f7d-4100-8b5e-5e868343b1dc None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:40.394485 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae3d3e-bb1a-4d53-81b8-f0916de2aaa7 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:40.445520 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0af66-3949-446c-8c9f-4a2ebea24442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:40.446593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0af66-3949-446c-8c9f-4a2ebea24442 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:40.447072 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0af66-3949-446c-8c9f-4a2ebea24442 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:40.447546 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0af66-3949-446c-8c9f-4a2ebea24442 None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:40.449254 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0af66-3949-446c-8c9f-4a2ebea24442 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:40.449858 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0af66-3949-446c-8c9f-4a2ebea24442 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:40.453671 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0af66-3949-446c-8c9f-4a2ebea24442 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:40.464516 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 388/787] 10.222.0.44 () {56 vars in 1124 bytes} [Sun Sep 11 14:59:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:40.466800 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0af66-3949-446c-8c9f-4a2ebea24442 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:40.484501 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:40.515833 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:40.609522 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cd533d-7f7d-4100-8b5e-5e868343b1dc None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:40.610115 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0af66-3949-446c-8c9f-4a2ebea24442 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:40.652212 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30158fd-34d4-4571-859f-892a234c0b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:40.653247 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30158fd-34d4-4571-859f-892a234c0b49 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:40.653733 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30158fd-34d4-4571-859f-892a234c0b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:40.654229 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30158fd-34d4-4571-859f-892a234c0b49 None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:40.655221 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30158fd-34d4-4571-859f-892a234c0b49 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:40.655776 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30158fd-34d4-4571-859f-892a234c0b49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:40.657458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30158fd-34d4-4571-859f-892a234c0b49 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:40.672712 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 389/788] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:59:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:40.674178 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30158fd-34d4-4571-859f-892a234c0b49 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:40.693844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e00b14-63d2-4cc5-9240-eda7d34cefba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:40.695629 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e00b14-63d2-4cc5-9240-eda7d34cefba None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:40.696468 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e00b14-63d2-4cc5-9240-eda7d34cefba None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:40.697028 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e00b14-63d2-4cc5-9240-eda7d34cefba None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:40.719910 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e00b14-63d2-4cc5-9240-eda7d34cefba None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:40.723103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e00b14-63d2-4cc5-9240-eda7d34cefba None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:40.737235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-18e00b14-63d2-4cc5-9240-eda7d34cefba None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 14:59:40.737235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:40.737235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:40.737235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:40.737235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:40.737235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:40.737235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:40.737235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:40.737235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:40.737235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:40.737235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:40.737235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:40.737235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:59:40.737235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:59:40.737235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:59:40.737235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:59:40.738346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:40.738346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:40.738346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:59:40.738346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:40.738346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:59:40.738346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:40.738346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:40.738346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:40.738346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:40.738346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:40.738346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:40.738346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:40.738346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:40.738346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:40.738346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:40.738346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:40.738346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:40.738346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:40.739277 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:40.739277 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:40.739277 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:59:40.739277 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:59:40.739277 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:59:40.739277 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:59:40.739277 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:59:40.739277 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:59:40.739277 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 14:59:40.739277 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:40.741818 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 400/789] 10.222.0.44 () {56 vars in 1117 bytes} [Sun Sep 11 14:59:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:40.743107 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e00b14-63d2-4cc5-9240-eda7d34cefba None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:40.793264 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:40.847999 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30158fd-34d4-4571-859f-892a234c0b49 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:40.890434 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e00b14-63d2-4cc5-9240-eda7d34cefba None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:40.912400 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:40.933320 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a139b6cb-f2c5-45f6-b1f7-17726cf675ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:40.934989 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139b6cb-f2c5-45f6-b1f7-17726cf675ac None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:40.935453 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139b6cb-f2c5-45f6-b1f7-17726cf675ac None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:40.935901 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139b6cb-f2c5-45f6-b1f7-17726cf675ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:40.937780 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a139b6cb-f2c5-45f6-b1f7-17726cf675ac None admin] RBAC: Adding query filter params (interface=public, service_id=cd8744e59d8f448cab8cbf601b3feef0, region_id=RegionOne) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:40.938342 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a139b6cb-f2c5-45f6-b1f7-17726cf675ac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:40.940431 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41ae08-75e7-439d-8e2b-a03877aacfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:40.941290 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a139b6cb-f2c5-45f6-b1f7-17726cf675ac None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:40.941688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41ae08-75e7-439d-8e2b-a03877aacfb4 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:40.942166 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41ae08-75e7-439d-8e2b-a03877aacfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:40.942672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41ae08-75e7-439d-8e2b-a03877aacfb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:40.955409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 390/790] 10.222.0.44 () {56 vars in 1251 bytes} [Sun Sep 11 14:59:40 2022] GET /identity/v3/endpoints?service_id=cd8744e59d8f448cab8cbf601b3feef0&interface=public®ion_id=RegionOne => generated 194 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:40.957989 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a139b6cb-f2c5-45f6-b1f7-17726cf675ac None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:40.967608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41ae08-75e7-439d-8e2b-a03877aacfb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:40.969648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41ae08-75e7-439d-8e2b-a03877aacfb4 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:40.988604 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-0e41ae08-75e7-439d-8e2b-a03877aacfb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:40.988604 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:40.988604 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:40.988604 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:40.988604 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:40.988604 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:40.988604 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:40.988604 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:40.988604 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:40.988604 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:40.988604 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:40.988604 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:40.988604 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:40.988604 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:40.988604 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:40.988604 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:40.989909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:40.989909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:40.989909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:40.989909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:40.989909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:40.989909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:40.989909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:40.989909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:40.989909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:40.989909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:40.989909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:40.989909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:40.989909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:40.989909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:40.989909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:40.989909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:40.989909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:40.990846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:40.990846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:40.990846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:40.997319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 401/791] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:40.998706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41ae08-75e7-439d-8e2b-a03877aacfb4 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:41.123639 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a139b6cb-f2c5-45f6-b1f7-17726cf675ac None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:41.144502 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41ae08-75e7-439d-8e2b-a03877aacfb4 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:41.162629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:41.168435 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60a331-2c3b-44b9-998d-2d2834d28406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:41.169089 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60a331-2c3b-44b9-998d-2d2834d28406 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:41.169377 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60a331-2c3b-44b9-998d-2d2834d28406 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:41.169642 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60a331-2c3b-44b9-998d-2d2834d28406 None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:41.170401 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f60a331-2c3b-44b9-998d-2d2834d28406 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:41.170702 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f60a331-2c3b-44b9-998d-2d2834d28406 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:41.172193 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f60a331-2c3b-44b9-998d-2d2834d28406 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:41.195439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed57de-c392-46ae-8003-c8f379f75188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:41.196527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed57de-c392-46ae-8003-c8f379f75188 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:41.202133 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed57de-c392-46ae-8003-c8f379f75188 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:41.202333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 391/792] 10.222.0.44 () {56 vars in 1149 bytes} [Sun Sep 11 14:59:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:41.202416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed57de-c392-46ae-8003-c8f379f75188 None admin] PATH_INFO: `/v3/services` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:41.202515 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60a331-2c3b-44b9-998d-2d2834d28406 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:41.202599 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed57de-c392-46ae-8003-c8f379f75188 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:41.202599 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed57de-c392-46ae-8003-c8f379f75188 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:41.221355 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 402/793] 10.222.0.44 () {56 vars in 1087 bytes} [Sun Sep 11 14:59:40 2022] GET /identity/v3/services => generated 772 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:41.221355 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94460e57-50ca-4e3e-98d7-881a63d745fe None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:41.221355 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94460e57-50ca-4e3e-98d7-881a63d745fe None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:41.221355 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94460e57-50ca-4e3e-98d7-881a63d745fe None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:41.221355 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 403/794] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:41.221355 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94460e57-50ca-4e3e-98d7-881a63d745fe None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:41.319571 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:41.363993 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60a331-2c3b-44b9-998d-2d2834d28406 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:41.408214 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf5f06-83a3-448e-879f-f3f5f9508b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:41.408851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf5f06-83a3-448e-879f-f3f5f9508b6c None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:41.409745 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf5f06-83a3-448e-879f-f3f5f9508b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:41.410015 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf5f06-83a3-448e-879f-f3f5f9508b6c None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:41.410697 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf5f06-83a3-448e-879f-f3f5f9508b6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:41.411178 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf5f06-83a3-448e-879f-f3f5f9508b6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:41.412869 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf5f06-83a3-448e-879f-f3f5f9508b6c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:41.432553 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 392/795] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:41.432553 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ba53c-efab-4905-a339-cbfd9fd9cc83 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:41.432553 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ba53c-efab-4905-a339-cbfd9fd9cc83 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:41.432553 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ba53c-efab-4905-a339-cbfd9fd9cc83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:41.446181 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94460e57-50ca-4e3e-98d7-881a63d745fe None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:41.454238 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-ed7ba53c-efab-4905-a339-cbfd9fd9cc83 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:41.455559 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-ed7ba53c-efab-4905-a339-cbfd9fd9cc83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:41.500203 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd5f09-58cc-48e9-8b94-3dce51e82f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:41.501220 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd5f09-58cc-48e9-8b94-3dce51e82f8a None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:41.501701 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd5f09-58cc-48e9-8b94-3dce51e82f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:41.502139 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd5f09-58cc-48e9-8b94-3dce51e82f8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:41.512822 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:41.515081 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd5f09-58cc-48e9-8b94-3dce51e82f8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:41.517011 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd5f09-58cc-48e9-8b94-3dce51e82f8a None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:41.532331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-acbd5f09-58cc-48e9-8b94-3dce51e82f8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:41.532331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:41.532331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:41.532331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:41.532331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:41.532331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:41.532331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:41.532331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:41.532331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:41.532331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:41.532331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:41.532331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:41.532331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:41.532331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:41.532331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:41.532331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:41.534063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:41.534063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:41.534063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:41.534063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:41.534063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:41.534063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:41.534063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:41.534063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:41.534063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:41.534063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:41.534063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:41.534063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:41.534063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:41.534063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:41.534063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:41.534063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:41.534063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:41.535491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:41.535491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:41.535491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:41.535491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 404/796] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:41.535491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af3919-e32b-4828-aa95-ae386fbaecb9 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:41.535491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af3919-e32b-4828-aa95-ae386fbaecb9 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:41.535491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af3919-e32b-4828-aa95-ae386fbaecb9 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:41.536927 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 405/797] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:41.541572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af3919-e32b-4828-aa95-ae386fbaecb9 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:41.546657 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 393/798] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:59:41.547610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ba53c-efab-4905-a339-cbfd9fd9cc83 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:41.683051 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:41.712944 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af3919-e32b-4828-aa95-ae386fbaecb9 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:41.721524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ba53c-efab-4905-a339-cbfd9fd9cc83 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:41.761506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a679896-9ae8-4a9b-9d85-ac18d58ea706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:41.762133 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a679896-9ae8-4a9b-9d85-ac18d58ea706 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:41.762446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a679896-9ae8-4a9b-9d85-ac18d58ea706 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:41.762711 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a679896-9ae8-4a9b-9d85-ac18d58ea706 None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:41.763361 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a679896-9ae8-4a9b-9d85-ac18d58ea706 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:41.763655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a679896-9ae8-4a9b-9d85-ac18d58ea706 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:41.765099 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a679896-9ae8-4a9b-9d85-ac18d58ea706 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:41.787944 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 394/799] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:41.788656 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fdfcb3-ed91-4a45-9c7d-fbd1c7a3dae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:41.789333 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdfcb3-ed91-4a45-9c7d-fbd1c7a3dae8 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:41.789621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdfcb3-ed91-4a45-9c7d-fbd1c7a3dae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:41.789875 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdfcb3-ed91-4a45-9c7d-fbd1c7a3dae8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:41.792032 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a17e2-c58d-473e-8be8-18179ccfc789 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:41.793461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a17e2-c58d-473e-8be8-18179ccfc789 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:41.795072 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a17e2-c58d-473e-8be8-18179ccfc789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:41.800246 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fdfcb3-ed91-4a45-9c7d-fbd1c7a3dae8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:41.801930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fdfcb3-ed91-4a45-9c7d-fbd1c7a3dae8 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:41.816573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-d0fdfcb3-ed91-4a45-9c7d-fbd1c7a3dae8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 14:59:41.816573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:41.816573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:41.816573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:41.816573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:41.816573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:41.816573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:41.816573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:41.816573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:41.816573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:41.816573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:41.816573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:41.816573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:59:41.816573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:59:41.816573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:59:41.816573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:59:41.818382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:41.818382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:41.818382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:59:41.818382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:41.818382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:59:41.818382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:41.818382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:41.818382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:41.818382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:41.818382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:41.818382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:41.818382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:41.818382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:41.818382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:41.818382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:41.818382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:41.818382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:41.818382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:41.820314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:41.820314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:41.820314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:59:41.820314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:59:41.820314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:59:41.820314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:59:41.820314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:59:41.820314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:59:41.820314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 14:59:41.820314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:41.822842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 406/800] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 14:59:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:41.824008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fdfcb3-ed91-4a45-9c7d-fbd1c7a3dae8 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:41.836896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-a26a17e2-c58d-473e-8be8-18179ccfc789 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:41.838462 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-a26a17e2-c58d-473e-8be8-18179ccfc789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:41.881385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:41.921768 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 395/801] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:59:41.929893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a17e2-c58d-473e-8be8-18179ccfc789 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:42.067389 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fdfcb3-ed91-4a45-9c7d-fbd1c7a3dae8 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:42.120494 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61216506-7ecc-416b-a59c-8428cedafe6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:42.121531 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61216506-7ecc-416b-a59c-8428cedafe6d None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:42.121759 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61216506-7ecc-416b-a59c-8428cedafe6d None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:42.122116 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61216506-7ecc-416b-a59c-8428cedafe6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:42.134271 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61216506-7ecc-416b-a59c-8428cedafe6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:42.136463 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61216506-7ecc-416b-a59c-8428cedafe6d None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:42.140056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a17e2-c58d-473e-8be8-18179ccfc789 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:42.154737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-61216506-7ecc-416b-a59c-8428cedafe6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:42.154737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:42.154737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:42.154737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:42.154737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:42.154737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:42.154737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:42.154737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:42.154737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:42.154737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:42.154737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:42.154737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:42.154737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:59:42.154737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:59:42.154737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:59:42.156432 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:59:42.156432 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:42.156432 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:42.156432 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:42.156432 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:42.156432 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:42.156432 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:42.156432 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:42.156432 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:42.156432 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:42.156432 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:42.156432 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:42.156432 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:42.156432 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:42.156432 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:42.156432 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:42.156432 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:42.158031 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:59:42.158031 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:59:42.158031 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:59:42.158031 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:59:42.158031 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:59:42.158031 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:59:42.158031 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:42.158031 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:42.158031 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 407/802] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:59:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:42.158995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61216506-7ecc-416b-a59c-8428cedafe6d None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:42.176653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:42.212340 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fae9f0-f5f8-4be8-8dbb-66f8e9b8def4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:42.213686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fae9f0-f5f8-4be8-8dbb-66f8e9b8def4 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:42.214489 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fae9f0-f5f8-4be8-8dbb-66f8e9b8def4 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:42.215167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fae9f0-f5f8-4be8-8dbb-66f8e9b8def4 None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:42.216712 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fae9f0-f5f8-4be8-8dbb-66f8e9b8def4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:42.217497 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fae9f0-f5f8-4be8-8dbb-66f8e9b8def4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:42.220366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fae9f0-f5f8-4be8-8dbb-66f8e9b8def4 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:42.254583 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 396/803] 10.222.0.44 () {56 vars in 1135 bytes} [Sun Sep 11 14:59:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:42.256259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fae9f0-f5f8-4be8-8dbb-66f8e9b8def4 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:42.291529 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:42.366969 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61216506-7ecc-416b-a59c-8428cedafe6d None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:42.415814 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fae9f0-f5f8-4be8-8dbb-66f8e9b8def4 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:42.429522 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058dd379-1804-4081-9627-b9a264504071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:42.430507 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058dd379-1804-4081-9627-b9a264504071 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:42.430931 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058dd379-1804-4081-9627-b9a264504071 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:42.431354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058dd379-1804-4081-9627-b9a264504071 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:42.451927 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058dd379-1804-4081-9627-b9a264504071 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:59:42.460789 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-058dd379-1804-4081-9627-b9a264504071 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:59:42.460789 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:42.460789 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:42.460789 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:42.460789 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:42.460789 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:42.460789 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:42.460789 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:42.460789 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:42.460789 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:42.460789 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:42.460789 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:42.460789 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:59:42.460789 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:59:42.460789 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:59:42.460789 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:59:42.462863 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:59:42.462863 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:59:42.462863 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:59:42.462863 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:59:42.462863 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:42.462863 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:42.462863 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:42.462863 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:42.462863 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:42.462863 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:42.462863 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:42.462863 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:42.462863 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:42.462863 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:42.462863 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:42.462863 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:42.464469 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:42.464469 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:42.464469 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:42.464469 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:42.464469 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:59:42.464469 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:59:42.464469 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:59:42.464469 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:59:42.464469 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:59:42.464469 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:59:42.464469 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:59:42.464469 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:42.469675 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 408/804] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 14:59:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:42.472934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058dd379-1804-4081-9627-b9a264504071 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:42.476402 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cde132-687a-4e25-93a0-2565358be533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:42.477265 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cde132-687a-4e25-93a0-2565358be533 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:42.477705 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cde132-687a-4e25-93a0-2565358be533 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:42.478110 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cde132-687a-4e25-93a0-2565358be533 None admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:42.478985 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cde132-687a-4e25-93a0-2565358be533 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:42.479448 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cde132-687a-4e25-93a0-2565358be533 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:42.481282 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cde132-687a-4e25-93a0-2565358be533 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:42.499212 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 397/805] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 14:59:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:42.500705 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cde132-687a-4e25-93a0-2565358be533 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:42.609025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:42.629664 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:42.645277 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cde132-687a-4e25-93a0-2565358be533 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:42.678107 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058dd379-1804-4081-9627-b9a264504071 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:42.702623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c31d8-3856-4fa8-aa40-e68b6f00b362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:42.704022 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c31d8-3856-4fa8-aa40-e68b6f00b362 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:42.704686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c31d8-3856-4fa8-aa40-e68b6f00b362 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:42.705388 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c31d8-3856-4fa8-aa40-e68b6f00b362 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:42.726407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c31d8-3856-4fa8-aa40-e68b6f00b362 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:42.729487 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c31d8-3856-4fa8-aa40-e68b6f00b362 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:42.738116 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e9da2-7504-40d3-8f8a-12ed41db6b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:42.739177 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e9da2-7504-40d3-8f8a-12ed41db6b03 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:42.739651 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e9da2-7504-40d3-8f8a-12ed41db6b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:42.740122 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e9da2-7504-40d3-8f8a-12ed41db6b03 None admin] PATH_INFO: `/v3/services/network` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:42.741197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9e9da2-7504-40d3-8f8a-12ed41db6b03 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:42.743778 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9e9da2-7504-40d3-8f8a-12ed41db6b03 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:42.758728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-bf4c31d8-3856-4fa8-aa40-e68b6f00b362 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:42.758728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:42.758728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:42.758728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:42.758728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:42.758728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:42.758728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:42.758728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:42.758728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:42.758728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:42.758728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:42.758728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:42.758728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:42.758728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:42.758728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:42.758728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:42.759746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:42.759746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:42.759746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:42.759746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:42.759746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:42.759746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:42.759746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:42.759746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:42.759746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:42.759746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:42.759746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:42.759746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:42.759746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:42.759746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:42.759746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:42.759746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:42.759746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:42.760657 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:42.760657 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:42.760657 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:42.760829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-cf9e9da2-7504-40d3-8f8a-12ed41db6b03 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 14:59:42.760829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:42.760829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:59:42.760829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:59:42.760829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:59:42.760829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:59:42.760829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:59:42.760829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:59:42.760829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 14:59:42.760829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:42.760829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:59:42.760829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:42.760829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:42.760829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:42.760829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:42.760829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:42.760829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:42.762608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:42.762608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:42.762608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:42.762608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:42.762608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:42.762608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:42.762608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:59:42.762608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:59:42.762608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:59:42.762608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:59:42.762608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:42.762608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:42.762608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:42.762608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:42.762608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:42.762608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:42.763839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:42.763839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:42.763839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:42.763839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:42.763839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:42.763839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:42.763839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:42.763839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:42.763839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:42.763839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:42.763839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:59:42.763839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:59:42.763839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 14:59:42.763839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:42.764813 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 398/806] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:42.764813 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c31d8-3856-4fa8-aa40-e68b6f00b362 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:42.769666 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 409/807] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:59:42 2022] GET /identity/v3/services/network => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:42.770937 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e9da2-7504-40d3-8f8a-12ed41db6b03 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:42.897866 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c31d8-3856-4fa8-aa40-e68b6f00b362 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:42.938792 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e9da2-7504-40d3-8f8a-12ed41db6b03 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:42.959816 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6b65a-591d-4bfa-8bdc-05beea68e2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:42.960446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6b65a-591d-4bfa-8bdc-05beea68e2a7 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:42.960728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6b65a-591d-4bfa-8bdc-05beea68e2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:42.961455 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6b65a-591d-4bfa-8bdc-05beea68e2a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:42.961700 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb6b65a-591d-4bfa-8bdc-05beea68e2a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:42.962002 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb6b65a-591d-4bfa-8bdc-05beea68e2a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:42.963393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb6b65a-591d-4bfa-8bdc-05beea68e2a7 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:42.982531 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 399/808] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:59:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:42.982531 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6b65a-591d-4bfa-8bdc-05beea68e2a7 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:42.984556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:42.988750 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:42.993172 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64524a31-9dcd-4b42-b005-a96eb1b18c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:42.994335 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64524a31-9dcd-4b42-b005-a96eb1b18c44 None admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:42.994827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64524a31-9dcd-4b42-b005-a96eb1b18c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:42.995993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64524a31-9dcd-4b42-b005-a96eb1b18c44 None admin] PATH_INFO: `/v3/projects/70ac515938a24c45ae6302dbac46a1ab/users/3fada78ceaeb4f3bad77c516f6dfcfd4/roles/ea3b2f1eb283418b8823cb4fff80edba` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:43.024454 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64524a31-9dcd-4b42-b005-a96eb1b18c44 None admin] RBAC: Authorizing `identity:create_grant(project_id=70ac515938a24c45ae6302dbac46a1ab, user_id=3fada78ceaeb4f3bad77c516f6dfcfd4, role_id=ea3b2f1eb283418b8823cb4fff80edba)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:43.027192 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64524a31-9dcd-4b42-b005-a96eb1b18c44 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:43.056586 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 410/809] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 14:59:42 2022] PUT /identity/v3/projects/70ac515938a24c45ae6302dbac46a1ab/users/3fada78ceaeb4f3bad77c516f6dfcfd4/roles/ea3b2f1eb283418b8823cb4fff80edba => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:59:43.058624 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64524a31-9dcd-4b42-b005-a96eb1b18c44 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:43.144184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6b65a-591d-4bfa-8bdc-05beea68e2a7 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:43.220845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27515b4-d132-44c9-b949-c160271057f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:43.222665 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27515b4-d132-44c9-b949-c160271057f9 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:43.223332 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27515b4-d132-44c9-b949-c160271057f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:43.223998 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27515b4-d132-44c9-b949-c160271057f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:43.225228 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27515b4-d132-44c9-b949-c160271057f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:43.225821 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27515b4-d132-44c9-b949-c160271057f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:43.228390 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27515b4-d132-44c9-b949-c160271057f9 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:43.256739 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 400/810] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:43.258548 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27515b4-d132-44c9-b949-c160271057f9 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:43.285942 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64524a31-9dcd-4b42-b005-a96eb1b18c44 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:43.366544 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbb62c-2bf6-44a9-b4e6-68424aea2ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:43.368512 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbb62c-2bf6-44a9-b4e6-68424aea2ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:43.370284 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbb62c-2bf6-44a9-b4e6-68424aea2ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:43.371130 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbb62c-2bf6-44a9-b4e6-68424aea2ff0 None admin] PATH_INFO: `/v3/services` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:43.372891 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fbb62c-2bf6-44a9-b4e6-68424aea2ff0 None admin] RBAC: Adding query filter params (name=network) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:43.373867 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fbb62c-2bf6-44a9-b4e6-68424aea2ff0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:43.376928 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fbb62c-2bf6-44a9-b4e6-68424aea2ff0 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:43.386513 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 411/811] 10.222.0.44 () {56 vars in 1112 bytes} [Sun Sep 11 14:59:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:43.388472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbb62c-2bf6-44a9-b4e6-68424aea2ff0 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:43.406602 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:43.480520 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:43.484059 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27515b4-d132-44c9-b949-c160271057f9 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:43.552678 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a82b5e-44ac-40e8-9f2f-df8887a5d734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:43.553921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a82b5e-44ac-40e8-9f2f-df8887a5d734 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:43.554397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a82b5e-44ac-40e8-9f2f-df8887a5d734 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:43.554864 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a82b5e-44ac-40e8-9f2f-df8887a5d734 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:43.575923 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a82b5e-44ac-40e8-9f2f-df8887a5d734 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:43.578835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a82b5e-44ac-40e8-9f2f-df8887a5d734 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:43.591530 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbb62c-2bf6-44a9-b4e6-68424aea2ff0 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:43.606693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-b2a82b5e-44ac-40e8-9f2f-df8887a5d734 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:43.606693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:43.606693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:43.606693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:43.606693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:43.606693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:43.606693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:43.606693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:43.606693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:43.606693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:43.606693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:43.606693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:43.606693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:43.606693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:43.606693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:43.606693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:43.608473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:43.608473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:43.608473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:43.608473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:43.608473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:43.608473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:43.608473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:43.608473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:43.608473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:43.608473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:43.608473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:43.608473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:43.608473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:43.608473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:43.608473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:43.608473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:43.608473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:43.610189 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:43.610189 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:43.610189 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:43.616380 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 401/812] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:43.618126 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a82b5e-44ac-40e8-9f2f-df8887a5d734 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:43.635225 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb47af6-405f-4641-8bfc-5ae09c87a038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:43.636102 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb47af6-405f-4641-8bfc-5ae09c87a038 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:43.636474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb47af6-405f-4641-8bfc-5ae09c87a038 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:43.636844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb47af6-405f-4641-8bfc-5ae09c87a038 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:43.644597 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb47af6-405f-4641-8bfc-5ae09c87a038 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:43.646441 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb47af6-405f-4641-8bfc-5ae09c87a038 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:43.659496 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-8cb47af6-405f-4641-8bfc-5ae09c87a038 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:43.659496 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:43.659496 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:43.659496 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:43.659496 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:43.659496 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:43.659496 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:43.659496 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:43.659496 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:43.659496 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:43.659496 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:43.659496 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:43.659496 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:59:43.659496 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:59:43.659496 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:59:43.661457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:59:43.661457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:43.661457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:43.661457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:43.661457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:43.661457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:43.661457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:43.661457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:43.661457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:43.661457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:43.661457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:43.661457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:43.661457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:43.661457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:43.661457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:43.661457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:43.661457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:43.663537 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:59:43.663537 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:59:43.663537 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:59:43.663537 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:59:43.663537 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:59:43.663537 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:59:43.663537 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:43.663537 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:43.663537 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 412/813] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:59:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:43.664935 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb47af6-405f-4641-8bfc-5ae09c87a038 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:43.793001 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb47af6-405f-4641-8bfc-5ae09c87a038 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:43.796055 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:43.842386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f6dd9-de41-416b-8690-baa3e8a6fa3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:43.843498 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f6dd9-de41-416b-8690-baa3e8a6fa3a None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:43.843985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f6dd9-de41-416b-8690-baa3e8a6fa3a None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:43.844491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f6dd9-de41-416b-8690-baa3e8a6fa3a None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:43.845536 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f6dd9-de41-416b-8690-baa3e8a6fa3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:43.846423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f6dd9-de41-416b-8690-baa3e8a6fa3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:43.848176 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f6dd9-de41-416b-8690-baa3e8a6fa3a None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:43.848428 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a82b5e-44ac-40e8-9f2f-df8887a5d734 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:43.876102 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 413/814] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:43.876102 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f6dd9-de41-416b-8690-baa3e8a6fa3a None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:43.880040 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:43.905023 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240f310-4aea-49f6-a195-953726319ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:43.906090 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240f310-4aea-49f6-a195-953726319ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:43.906600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240f310-4aea-49f6-a195-953726319ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:43.907158 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240f310-4aea-49f6-a195-953726319ea0 None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:43.908004 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240f310-4aea-49f6-a195-953726319ea0 None admin] RBAC: Adding query filter params (type=network) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:43.908579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240f310-4aea-49f6-a195-953726319ea0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:43.910299 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240f310-4aea-49f6-a195-953726319ea0 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:43.921737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 402/815] 10.222.0.44 () {56 vars in 1112 bytes} [Sun Sep 11 14:59:43 2022] GET /identity/v3/services?type=network => generated 350 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:43.923071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240f310-4aea-49f6-a195-953726319ea0 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:44.027750 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f6dd9-de41-416b-8690-baa3e8a6fa3a None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:44.054560 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240f310-4aea-49f6-a195-953726319ea0 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:44.067443 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14822b32-dc1a-477e-bc18-de847850e191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:44.068065 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14822b32-dc1a-477e-bc18-de847850e191 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:44.068340 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14822b32-dc1a-477e-bc18-de847850e191 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:44.068613 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14822b32-dc1a-477e-bc18-de847850e191 None admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:44.069348 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14822b32-dc1a-477e-bc18-de847850e191 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:44.069718 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14822b32-dc1a-477e-bc18-de847850e191 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:44.071189 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14822b32-dc1a-477e-bc18-de847850e191 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:44.090206 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 414/816] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 14:59:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:44.090206 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14822b32-dc1a-477e-bc18-de847850e191 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:44.101203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726b9f0-9360-4afc-a7bb-29e1786f74d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:44.101836 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726b9f0-9360-4afc-a7bb-29e1786f74d7 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:44.102113 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726b9f0-9360-4afc-a7bb-29e1786f74d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:44.102382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726b9f0-9360-4afc-a7bb-29e1786f74d7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:44.111215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f726b9f0-9360-4afc-a7bb-29e1786f74d7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:44.112691 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f726b9f0-9360-4afc-a7bb-29e1786f74d7 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:44.120875 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:44.125617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-f726b9f0-9360-4afc-a7bb-29e1786f74d7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:59:44.125617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:44.125617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:44.125617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:44.125617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:44.125617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:44.125617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:44.125617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:44.125617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:44.125617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:44.125617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:44.125617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:44.125617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:59:44.125617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:59:44.125617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:59:44.125617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:59:44.126589 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:44.126589 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:44.126589 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:59:44.126589 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:44.126589 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:59:44.126589 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:44.126589 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:44.126589 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:44.126589 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:44.126589 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:44.126589 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:44.126589 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:44.126589 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:44.126589 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:44.126589 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:44.126589 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:44.126589 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:44.126589 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:44.127571 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:44.127571 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:44.127571 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:59:44.127571 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:59:44.127571 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:59:44.127571 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:59:44.127571 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:59:44.127571 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:59:44.127571 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 14:59:44.127571 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:44.127571 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 403/817] 10.222.0.44 () {56 vars in 1093 bytes} [Sun Sep 11 14:59:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:44.128159 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fef8f2-71e9-4a51-b1cd-dad3a9f8a051 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:44.128226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fef8f2-71e9-4a51-b1cd-dad3a9f8a051 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:44.128669 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fef8f2-71e9-4a51-b1cd-dad3a9f8a051 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:44.134368 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 404/818] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:44.134368 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fef8f2-71e9-4a51-b1cd-dad3a9f8a051 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:44.178339 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:44.251106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14822b32-dc1a-477e-bc18-de847850e191 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:44.277250 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fef8f2-71e9-4a51-b1cd-dad3a9f8a051 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:44.307307 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81445e03-742e-458e-92bf-c303ef0f9d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:44.308119 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81445e03-742e-458e-92bf-c303ef0f9d30 None admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:44.308290 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81445e03-742e-458e-92bf-c303ef0f9d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:44.308730 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81445e03-742e-458e-92bf-c303ef0f9d30 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:44.309963 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81445e03-742e-458e-92bf-c303ef0f9d30 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:44.312341 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81445e03-742e-458e-92bf-c303ef0f9d30 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:44.333847 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de21379-eac7-4a1e-90db-b7357e98186a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:44.335033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de21379-eac7-4a1e-90db-b7357e98186a None admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:44.335608 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de21379-eac7-4a1e-90db-b7357e98186a None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:44.336121 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de21379-eac7-4a1e-90db-b7357e98186a None admin] PATH_INFO: `/v3/projects/70ac515938a24c45ae6302dbac46a1ab/users/05aa5e40ee4542719657bc03b2015e72/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:44.339348 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de21379-eac7-4a1e-90db-b7357e98186a None admin] RBAC: Authorizing `identity:create_grant(project_id=70ac515938a24c45ae6302dbac46a1ab, user_id=05aa5e40ee4542719657bc03b2015e72, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:44.341709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de21379-eac7-4a1e-90db-b7357e98186a None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:44.349077 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 415/819] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 14:59:44 2022] POST /identity/v3/endpoints => generated 337 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:59:44.350820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088b41c-06b4-4bdc-98f9-bd1ef71e4341 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:44.351220 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088b41c-06b4-4bdc-98f9-bd1ef71e4341 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:44.351590 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088b41c-06b4-4bdc-98f9-bd1ef71e4341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:44.373810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 405/820] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 14:59:44 2022] PUT /identity/v3/projects/70ac515938a24c45ae6302dbac46a1ab/users/05aa5e40ee4542719657bc03b2015e72/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 14:59:44.375320 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de21379-eac7-4a1e-90db-b7357e98186a None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:44.390460 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-3088b41c-06b4-4bdc-98f9-bd1ef71e4341 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:44.391755 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-3088b41c-06b4-4bdc-98f9-bd1ef71e4341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:44.600891 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:44.605649 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 416/821] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:59:44.613220 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de21379-eac7-4a1e-90db-b7357e98186a None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:44.683384 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e7b93-aca2-4bb1-9d39-dcc46e133e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:44.684420 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e7b93-aca2-4bb1-9d39-dcc46e133e59 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:44.684960 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e7b93-aca2-4bb1-9d39-dcc46e133e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:44.685421 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e7b93-aca2-4bb1-9d39-dcc46e133e59 None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:44.686898 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e7b93-aca2-4bb1-9d39-dcc46e133e59 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:44.687450 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e7b93-aca2-4bb1-9d39-dcc46e133e59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:44.690050 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e7b93-aca2-4bb1-9d39-dcc46e133e59 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:44.725316 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 406/822] 10.222.0.44 () {56 vars in 1133 bytes} [Sun Sep 11 14:59:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:44.731600 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3088b41c-06b4-4bdc-98f9-bd1ef71e4341 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:44.747205 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:44.937836 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3088b41c-06b4-4bdc-98f9-bd1ef71e4341 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:44.969670 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e7b93-aca2-4bb1-9d39-dcc46e133e59 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:44.998900 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ce79f2-7a37-4fff-852a-7988b7d0403e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:45.002406 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce79f2-7a37-4fff-852a-7988b7d0403e None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:45.002736 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce79f2-7a37-4fff-852a-7988b7d0403e None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:45.003007 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce79f2-7a37-4fff-852a-7988b7d0403e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:45.150417 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e7b93-aca2-4bb1-9d39-dcc46e133e59 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:45.163667 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ce79f2-7a37-4fff-852a-7988b7d0403e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:45.167858 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ce79f2-7a37-4fff-852a-7988b7d0403e None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:45.179336 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:45.195168 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9e154-1f6f-45b0-a0c5-17b98a7f8040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:45.195851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9e154-1f6f-45b0-a0c5-17b98a7f8040 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:45.196048 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9e154-1f6f-45b0-a0c5-17b98a7f8040 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:45.196317 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9e154-1f6f-45b0-a0c5-17b98a7f8040 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:45.196910 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a9e154-1f6f-45b0-a0c5-17b98a7f8040 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:45.198374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a9e154-1f6f-45b0-a0c5-17b98a7f8040 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:45.211509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-77a9e154-1f6f-45b0-a0c5-17b98a7f8040 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 14:59:45.211509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:45.211509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:59:45.211509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:59:45.211509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:59:45.211509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:59:45.211509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:59:45.211509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:59:45.211509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 14:59:45.211509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:45.211509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:59:45.211509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:45.211509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:45.211509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:45.211509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:45.211509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:45.211509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:45.212599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:45.212599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:45.212599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:45.212599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:45.212599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:45.212599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:45.212599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:59:45.212599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:59:45.212599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:59:45.212599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:59:45.212599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:45.212599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:45.212599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:45.212599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:45.212599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:45.212599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:45.214023 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:45.214023 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:45.214023 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:45.214023 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:45.214023 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:45.214023 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:45.214023 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:45.214023 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:45.214023 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:45.214023 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:45.214023 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:59:45.214023 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:59:45.214023 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 14:59:45.214023 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:45.214023 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 407/823] 10.222.0.44 () {56 vars in 1129 bytes} [Sun Sep 11 14:59:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:45.222512 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9e154-1f6f-45b0-a0c5-17b98a7f8040 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:45.253526 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:45.326561 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 417/824] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:59:44 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 14:59:45.335127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ce79f2-7a37-4fff-852a-7988b7d0403e None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:45.364224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9e154-1f6f-45b0-a0c5-17b98a7f8040 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:45.403797 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b657c-d216-4310-a661-a69698c70206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:45.404949 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b657c-d216-4310-a661-a69698c70206 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:45.405449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b657c-d216-4310-a661-a69698c70206 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:45.405940 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b657c-d216-4310-a661-a69698c70206 None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:45.406781 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b657c-d216-4310-a661-a69698c70206 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:45.407327 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b657c-d216-4310-a661-a69698c70206 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:45.409093 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b657c-d216-4310-a661-a69698c70206 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:45.415260 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 408/825] 10.222.0.44 () {56 vars in 1124 bytes} [Sun Sep 11 14:59:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:45.424427 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b657c-d216-4310-a661-a69698c70206 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:45.441006 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ce79f2-7a37-4fff-852a-7988b7d0403e None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:45.456588 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:45.482717 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1f9d5-1f7b-479e-bf1a-08977e84fb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:45.483361 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1f9d5-1f7b-479e-bf1a-08977e84fb46 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:45.483634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1f9d5-1f7b-479e-bf1a-08977e84fb46 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:45.483896 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1f9d5-1f7b-479e-bf1a-08977e84fb46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:45.496485 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e1f9d5-1f7b-479e-bf1a-08977e84fb46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:45.498009 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e1f9d5-1f7b-479e-bf1a-08977e84fb46 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:45.518908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-a7e1f9d5-1f7b-479e-bf1a-08977e84fb46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:45.518908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:45.518908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:45.518908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:45.518908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:45.518908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:45.518908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:45.518908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:45.518908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:45.518908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:45.518908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:45.518908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:45.518908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:45.518908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:45.518908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:45.518908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:45.519923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:45.519923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:45.519923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:45.519923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:45.519923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:45.519923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:45.519923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:45.519923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:45.519923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:45.519923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:45.519923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:45.519923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:45.519923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:45.519923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:45.519923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:45.519923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:45.519923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:45.520809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:45.520809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:45.520809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:45.525576 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 418/826] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:45.529163 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1f9d5-1f7b-479e-bf1a-08977e84fb46 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:45.541802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:45.563872 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b657c-d216-4310-a661-a69698c70206 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:45.609810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79189a1c-91ab-4c5d-8fb6-9626d63d5df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:45.610452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79189a1c-91ab-4c5d-8fb6-9626d63d5df4 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:45.610731 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79189a1c-91ab-4c5d-8fb6-9626d63d5df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:45.610992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79189a1c-91ab-4c5d-8fb6-9626d63d5df4 None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:45.611651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79189a1c-91ab-4c5d-8fb6-9626d63d5df4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:45.611950 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79189a1c-91ab-4c5d-8fb6-9626d63d5df4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:45.613626 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79189a1c-91ab-4c5d-8fb6-9626d63d5df4 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:45.625384 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 409/827] 10.222.0.44 () {56 vars in 1124 bytes} [Sun Sep 11 14:59:45 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:45.627124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79189a1c-91ab-4c5d-8fb6-9626d63d5df4 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:45.776821 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79189a1c-91ab-4c5d-8fb6-9626d63d5df4 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:45.808392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1f9d5-1f7b-479e-bf1a-08977e84fb46 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:45.816295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc418823-fca4-436d-9acb-e17984013de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:45.816919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc418823-fca4-436d-9acb-e17984013de0 None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:45.817220 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc418823-fca4-436d-9acb-e17984013de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:45.817489 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc418823-fca4-436d-9acb-e17984013de0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:45.818166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc418823-fca4-436d-9acb-e17984013de0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:45.819584 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc418823-fca4-436d-9acb-e17984013de0 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:45.848765 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 410/828] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 14:59:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 14:59:45.859508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b435a9-9557-4050-ac3b-81d4c2620e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:45.860157 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b435a9-9557-4050-ac3b-81d4c2620e01 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:45.860429 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b435a9-9557-4050-ac3b-81d4c2620e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:45.860753 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b435a9-9557-4050-ac3b-81d4c2620e01 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:45.861491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b435a9-9557-4050-ac3b-81d4c2620e01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:45.861846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b435a9-9557-4050-ac3b-81d4c2620e01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:45.863391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b435a9-9557-4050-ac3b-81d4c2620e01 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:45.879153 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 419/829] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:45.885361 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc418823-fca4-436d-9acb-e17984013de0 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:45.889913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:46.149079 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc418823-fca4-436d-9acb-e17984013de0 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:46.192330 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1abb21-a9b9-4200-8fba-f5fe2db78c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:46.192996 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1abb21-a9b9-4200-8fba-f5fe2db78c1d None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:46.193309 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1abb21-a9b9-4200-8fba-f5fe2db78c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:46.193572 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1abb21-a9b9-4200-8fba-f5fe2db78c1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:46.202022 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1abb21-a9b9-4200-8fba-f5fe2db78c1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:46.203518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1abb21-a9b9-4200-8fba-f5fe2db78c1d None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:46.216321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-cb1abb21-a9b9-4200-8fba-f5fe2db78c1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:46.216321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:46.216321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:46.216321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:46.216321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:46.216321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:46.216321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:46.216321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:46.216321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:46.216321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:46.216321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:46.216321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:46.216321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:59:46.216321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:59:46.216321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:59:46.217435 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:59:46.217435 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:46.217435 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:46.217435 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:46.217435 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:46.217435 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:46.217435 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:46.217435 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:46.217435 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:46.217435 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:46.217435 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:46.217435 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:46.217435 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:46.217435 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:46.217435 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:46.217435 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:46.217435 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:46.218324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:59:46.218324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:59:46.218324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:59:46.218324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:59:46.218324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:59:46.218324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:59:46.218324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:46.218324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:46.218324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 411/830] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:59:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:46.224621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b435a9-9557-4050-ac3b-81d4c2620e01 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:46.243263 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:46.336101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b435a9-9557-4050-ac3b-81d4c2620e01 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:46.374425 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18e3f20-4f8e-4e9c-af9c-0ec0f2b30149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:46.375087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e3f20-4f8e-4e9c-af9c-0ec0f2b30149 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:46.375362 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e3f20-4f8e-4e9c-af9c-0ec0f2b30149 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:46.375625 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e3f20-4f8e-4e9c-af9c-0ec0f2b30149 None admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:46.376327 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18e3f20-4f8e-4e9c-af9c-0ec0f2b30149 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:46.376621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18e3f20-4f8e-4e9c-af9c-0ec0f2b30149 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:46.378108 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18e3f20-4f8e-4e9c-af9c-0ec0f2b30149 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:46.391560 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 420/831] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 14:59:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:46.397697 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1abb21-a9b9-4200-8fba-f5fe2db78c1d None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:46.508120 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:46.560395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1abb21-a9b9-4200-8fba-f5fe2db78c1d None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:46.618078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b3e08c-fcc0-4cd2-98ca-3156325a3c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:46.618462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10ef64-9744-4574-b9fd-0d3b4562c948 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:46.618714 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3e08c-fcc0-4cd2-98ca-3156325a3c54 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:46.618881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10ef64-9744-4574-b9fd-0d3b4562c948 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:46.618976 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3e08c-fcc0-4cd2-98ca-3156325a3c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:46.619247 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3e08c-fcc0-4cd2-98ca-3156325a3c54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:46.619352 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10ef64-9744-4574-b9fd-0d3b4562c948 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:46.620000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b3e08c-fcc0-4cd2-98ca-3156325a3c54 None admin] RBAC: Adding query filter params (role.id=c426875a15834aa5b122e2a62267d15a, scope.project.id=70ac515938a24c45ae6302dbac46a1ab, user.id=453297dac64046d19fc1c331d2d85abc) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:46.620383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b3e08c-fcc0-4cd2-98ca-3156325a3c54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:46.621892 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b3e08c-fcc0-4cd2-98ca-3156325a3c54 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:46.628551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 421/832] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:46 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:46.629275 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 412/833] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:59:46 2022] GET /identity/v3/role_assignments?user.id=453297dac64046d19fc1c331d2d85abc&scope.project.id=70ac515938a24c45ae6302dbac46a1ab&role.id=c426875a15834aa5b122e2a62267d15a => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:46.631418 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2d41d-4578-429e-94e8-1097f83ce66f None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:46.631904 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2d41d-4578-429e-94e8-1097f83ce66f None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:46.632367 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2d41d-4578-429e-94e8-1097f83ce66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:46.663327 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-ecb2d41d-4578-429e-94e8-1097f83ce66f None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:46.665468 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-ecb2d41d-4578-429e-94e8-1097f83ce66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:46.797305 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 422/834] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:46 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 14:59:47.012514 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92ce98-0feb-45bd-bc06-87b42f981b43 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:47.013188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92ce98-0feb-45bd-bc06-87b42f981b43 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:47.013660 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92ce98-0feb-45bd-bc06-87b42f981b43 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:47.015510 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 413/835] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:47.021493 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75baf5-6a3b-4e2a-94de-c96cd4657398 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:47.022124 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75baf5-6a3b-4e2a-94de-c96cd4657398 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:47.022585 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75baf5-6a3b-4e2a-94de-c96cd4657398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:47.032525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:47.048296 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-df75baf5-6a3b-4e2a-94de-c96cd4657398 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:47.049591 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-df75baf5-6a3b-4e2a-94de-c96cd4657398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:47.149637 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 423/836] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:47 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:59:47.166839 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92ce98-0feb-45bd-bc06-87b42f981b43 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:47.296225 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92ce98-0feb-45bd-bc06-87b42f981b43 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:47.330427 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:47.339893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e30310-652a-4af9-9088-272466fd9cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:47.340535 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e30310-652a-4af9-9088-272466fd9cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:47.340807 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e30310-652a-4af9-9088-272466fd9cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:47.341068 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e30310-652a-4af9-9088-272466fd9cf4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:47.353454 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e30310-652a-4af9-9088-272466fd9cf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:59:47.358823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-55e30310-652a-4af9-9088-272466fd9cf4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:47.358823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:47.358823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:47.358823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:47.358823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:47.358823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:47.358823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:47.358823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:47.358823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:47.358823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:47.358823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:47.358823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:47.358823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:59:47.358823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:59:47.358823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:59:47.358823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:59:47.359769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:59:47.359769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:59:47.359769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:59:47.359769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:59:47.359769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:47.359769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:47.359769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:47.359769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:47.359769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:47.359769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:47.359769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:47.359769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:47.359769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:47.359769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:47.359769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:47.359769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:47.360610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:47.360610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:47.360610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:47.360610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:47.360610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:59:47.360610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:59:47.360610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:59:47.360610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:59:47.360610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:59:47.360610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:59:47.360610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 14:59:47.360610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:47.363376 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 414/837] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 14:59:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:59:47.371304 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75baf5-6a3b-4e2a-94de-c96cd4657398 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:47.439034 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e30310-652a-4af9-9088-272466fd9cf4 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:47.500967 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75baf5-6a3b-4e2a-94de-c96cd4657398 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:47.557755 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec0a5f-d250-46d4-80c9-8b95549b58ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:47.558385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec0a5f-d250-46d4-80c9-8b95549b58ff None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:47.558705 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec0a5f-d250-46d4-80c9-8b95549b58ff None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:47.559024 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec0a5f-d250-46d4-80c9-8b95549b58ff None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:47.559696 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec0a5f-d250-46d4-80c9-8b95549b58ff None admin] RBAC: Adding query filter params (name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:47.560158 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec0a5f-d250-46d4-80c9-8b95549b58ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:47.561917 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec0a5f-d250-46d4-80c9-8b95549b58ff None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:47.563434 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:47.574861 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 424/838] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 14:59:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:59:47.582717 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec0a5f-d250-46d4-80c9-8b95549b58ff None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:47.583385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e30310-652a-4af9-9088-272466fd9cf4 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:47.630628 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc14be2-cd56-40ad-8c77-1339cd484b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:47.631319 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc14be2-cd56-40ad-8c77-1339cd484b5a None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:47.631751 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc14be2-cd56-40ad-8c77-1339cd484b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:47.632134 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc14be2-cd56-40ad-8c77-1339cd484b5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:47.647503 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc14be2-cd56-40ad-8c77-1339cd484b5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:59:47.654295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-0fc14be2-cd56-40ad-8c77-1339cd484b5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:59:47.654295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:47.654295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:47.654295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:47.654295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:47.654295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:47.654295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:47.654295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:47.654295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:47.654295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:47.654295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:47.654295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:47.654295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:59:47.654295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:59:47.654295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:59:47.654295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:59:47.655952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:59:47.655952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:59:47.655952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:59:47.655952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:59:47.655952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:47.655952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:47.655952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:47.655952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:47.655952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:47.655952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:47.655952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:47.655952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:47.655952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:47.655952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:47.655952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:47.655952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:47.656997 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:47.656997 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:47.656997 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:47.656997 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:47.656997 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:59:47.656997 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:59:47.656997 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:59:47.656997 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:59:47.656997 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:59:47.656997 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:59:47.656997 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 14:59:47.656997 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:47.656997 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 415/839] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 14:59:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:59:47.664018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc14be2-cd56-40ad-8c77-1339cd484b5a None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:47.734663 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:47.753193 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec0a5f-d250-46d4-80c9-8b95549b58ff None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:47.794491 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc14be2-cd56-40ad-8c77-1339cd484b5a None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:47.817780 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22095d-975a-48be-8fff-2801ce039c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:47.818930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22095d-975a-48be-8fff-2801ce039c68 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:47.819684 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22095d-975a-48be-8fff-2801ce039c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:47.819958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22095d-975a-48be-8fff-2801ce039c68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:47.836400 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333154ff-ae56-4b2c-9187-54aca3a8341b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:47.837310 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333154ff-ae56-4b2c-9187-54aca3a8341b None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:47.837736 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333154ff-ae56-4b2c-9187-54aca3a8341b None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:47.838314 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333154ff-ae56-4b2c-9187-54aca3a8341b None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:47.839255 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333154ff-ae56-4b2c-9187-54aca3a8341b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:47.839714 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333154ff-ae56-4b2c-9187-54aca3a8341b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:47.841336 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333154ff-ae56-4b2c-9187-54aca3a8341b None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:47.842701 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc22095d-975a-48be-8fff-2801ce039c68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:47.846748 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc22095d-975a-48be-8fff-2801ce039c68 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:47.853281 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 416/840] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 14:59:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:47.866753 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333154ff-ae56-4b2c-9187-54aca3a8341b None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:47.880820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-dc22095d-975a-48be-8fff-2801ce039c68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:47.880820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:47.880820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:47.880820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:47.880820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:47.880820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:47.880820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:47.880820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:47.880820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:47.880820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:47.880820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:47.880820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:47.880820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:47.880820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:47.880820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:47.880820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:47.882403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:47.882403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:47.882403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:47.882403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:47.882403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:47.882403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:47.882403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:47.882403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:47.882403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:47.882403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:47.882403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:47.882403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:47.882403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:47.882403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:47.882403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:47.882403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:47.882403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:47.883397 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:47.883397 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:47.883397 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:47.891393 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 425/841] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:59:47.896459 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22095d-975a-48be-8fff-2801ce039c68 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:47.994854 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:48.023619 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333154ff-ae56-4b2c-9187-54aca3a8341b None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:48.036393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:48.063931 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c808e90-9a0d-4829-a27b-4ffa5fb98217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:48.065079 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c808e90-9a0d-4829-a27b-4ffa5fb98217 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:48.065584 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c808e90-9a0d-4829-a27b-4ffa5fb98217 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:48.066053 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c808e90-9a0d-4829-a27b-4ffa5fb98217 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:48.078419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c808e90-9a0d-4829-a27b-4ffa5fb98217 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:48.080239 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c808e90-9a0d-4829-a27b-4ffa5fb98217 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:48.093923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22095d-975a-48be-8fff-2801ce039c68 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:48.097194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-1c808e90-9a0d-4829-a27b-4ffa5fb98217 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:48.097194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:48.097194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:48.097194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:48.097194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:48.097194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:48.097194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:48.097194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:48.097194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:48.097194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:48.097194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:48.097194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:48.097194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:48.097194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:48.097194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:48.097194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:48.098139 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:48.098139 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:48.098139 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:48.098139 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:48.098139 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:48.098139 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:48.098139 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:48.098139 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:48.098139 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:48.098139 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:48.098139 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:48.098139 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:48.098139 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:48.098139 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:48.098139 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:48.098139 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:48.098139 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:48.099031 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:48.099031 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:48.099031 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:48.101271 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 417/842] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:48.113124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c808e90-9a0d-4829-a27b-4ffa5fb98217 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:48.164956 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d05c2-0fc7-40af-8950-d19a49042090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:48.166016 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d05c2-0fc7-40af-8950-d19a49042090 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:48.166481 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d05c2-0fc7-40af-8950-d19a49042090 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:48.166963 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d05c2-0fc7-40af-8950-d19a49042090 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:48.168353 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81d05c2-0fc7-40af-8950-d19a49042090 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:48.168851 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81d05c2-0fc7-40af-8950-d19a49042090 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:48.171548 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81d05c2-0fc7-40af-8950-d19a49042090 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:48.204653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 426/843] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:59:48.206872 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d05c2-0fc7-40af-8950-d19a49042090 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:48.336701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c808e90-9a0d-4829-a27b-4ffa5fb98217 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:48.402635 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb292e-2254-4456-b0a3-1cabdc14ca19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:48.403382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb292e-2254-4456-b0a3-1cabdc14ca19 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:48.403684 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb292e-2254-4456-b0a3-1cabdc14ca19 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:48.404022 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb292e-2254-4456-b0a3-1cabdc14ca19 None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:48.404766 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb292e-2254-4456-b0a3-1cabdc14ca19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:48.405173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb292e-2254-4456-b0a3-1cabdc14ca19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:48.406868 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb292e-2254-4456-b0a3-1cabdc14ca19 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:48.419620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d05c2-0fc7-40af-8950-d19a49042090 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:48.428547 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 418/844] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:59:48.428547 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8487c-1889-49d2-b8b0-8a43d6c15c83 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:48.428547 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8487c-1889-49d2-b8b0-8a43d6c15c83 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:48.428547 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8487c-1889-49d2-b8b0-8a43d6c15c83 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:48.428547 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 419/845] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:48.431496 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf8487c-1889-49d2-b8b0-8a43d6c15c83 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:48.444285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:48.461422 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:48.499672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb266f-7c10-4b2b-a2c2-a11ad3592279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:48.500776 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb266f-7c10-4b2b-a2c2-a11ad3592279 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:48.501381 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb266f-7c10-4b2b-a2c2-a11ad3592279 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:48.501869 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb266f-7c10-4b2b-a2c2-a11ad3592279 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:48.516384 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb266f-7c10-4b2b-a2c2-a11ad3592279 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:48.519238 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb266f-7c10-4b2b-a2c2-a11ad3592279 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:48.541438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-2aeb266f-7c10-4b2b-a2c2-a11ad3592279 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 14:59:48.541438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:48.541438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:48.541438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:48.541438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:48.541438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:48.541438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:48.541438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:48.541438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:48.541438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:48.541438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:48.541438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:48.541438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:59:48.541438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:59:48.541438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:59:48.541438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:59:48.542558 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:48.542558 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:48.542558 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:59:48.542558 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:48.542558 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:59:48.542558 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:48.542558 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:48.542558 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:48.542558 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:48.542558 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:48.542558 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:48.542558 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:48.542558 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:48.542558 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:48.542558 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:48.542558 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:48.542558 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:48.542558 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:48.543495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:48.543495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:48.543495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:59:48.543495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:59:48.543495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:59:48.543495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:59:48.543495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:59:48.543495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:59:48.543495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 14:59:48.543495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:48.544031 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 427/846] 10.222.0.44 () {56 vars in 1117 bytes} [Sun Sep 11 14:59:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:59:48.546020 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cd213-b3a6-4d30-9d23-9026519d04b8 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:48.546469 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cd213-b3a6-4d30-9d23-9026519d04b8 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:48.546949 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cd213-b3a6-4d30-9d23-9026519d04b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:48.578961 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-8b3cd213-b3a6-4d30-9d23-9026519d04b8 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:48.581065 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-8b3cd213-b3a6-4d30-9d23-9026519d04b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:48.581573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf8487c-1889-49d2-b8b0-8a43d6c15c83 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:48.626592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23e085-9661-4af8-a6b9-b7b11beb8f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:48.627716 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23e085-9661-4af8-a6b9-b7b11beb8f23 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:48.628187 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23e085-9661-4af8-a6b9-b7b11beb8f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:48.628682 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23e085-9661-4af8-a6b9-b7b11beb8f23 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:48.638885 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c23e085-9661-4af8-a6b9-b7b11beb8f23 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:48.640935 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c23e085-9661-4af8-a6b9-b7b11beb8f23 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:48.662043 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-5c23e085-9661-4af8-a6b9-b7b11beb8f23 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 14:59:48.662043 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:48.662043 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:48.662043 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:48.662043 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:48.662043 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:48.662043 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:48.662043 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:48.662043 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:48.662043 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:48.662043 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:48.662043 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:48.662043 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:59:48.662043 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:59:48.662043 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:59:48.662043 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:59:48.663356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:48.663356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:48.663356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:59:48.663356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:48.663356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:59:48.663356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:48.663356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:48.663356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:48.663356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:48.663356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:48.663356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:48.663356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:48.663356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:48.663356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:48.663356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:48.663356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:48.663356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:48.663356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:48.664883 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:48.664883 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:48.664883 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:59:48.664883 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:59:48.664883 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:59:48.664883 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:59:48.664883 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:59:48.664883 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:59:48.664883 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 14:59:48.664883 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:48.664883 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 420/847] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 14:59:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:48.667077 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23e085-9661-4af8-a6b9-b7b11beb8f23 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:48.695525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 428/848] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:48 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:59:48.696400 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3cd213-b3a6-4d30-9d23-9026519d04b8 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:48.818341 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:48.842212 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23e085-9661-4af8-a6b9-b7b11beb8f23 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:48.849934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:48.859341 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3cd213-b3a6-4d30-9d23-9026519d04b8 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:48.881425 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3e357-4117-41c7-b0eb-727563d8e25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:48.882062 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3e357-4117-41c7-b0eb-727563d8e25b None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:48.882348 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3e357-4117-41c7-b0eb-727563d8e25b None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:48.882611 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3e357-4117-41c7-b0eb-727563d8e25b None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:48.883372 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3e357-4117-41c7-b0eb-727563d8e25b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:48.883669 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3e357-4117-41c7-b0eb-727563d8e25b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:48.885211 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3e357-4117-41c7-b0eb-727563d8e25b None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:48.910658 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 421/849] 10.222.0.44 () {56 vars in 1149 bytes} [Sun Sep 11 14:59:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:48.911665 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3e357-4117-41c7-b0eb-727563d8e25b None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:48.936207 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab51763-310e-4d6b-af05-baac71ea9b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:48.937918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab51763-310e-4d6b-af05-baac71ea9b40 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:48.938870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab51763-310e-4d6b-af05-baac71ea9b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:48.939632 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab51763-310e-4d6b-af05-baac71ea9b40 None admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:48.941445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab51763-310e-4d6b-af05-baac71ea9b40 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:48.942358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab51763-310e-4d6b-af05-baac71ea9b40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:48.945716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab51763-310e-4d6b-af05-baac71ea9b40 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:48.994448 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 429/850] 10.222.0.44 () {56 vars in 1135 bytes} [Sun Sep 11 14:59:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:49.000609 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab51763-310e-4d6b-af05-baac71ea9b40 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:49.045072 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3e357-4117-41c7-b0eb-727563d8e25b None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:49.086768 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b089a-1239-452c-9cb1-57311fbc995b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:49.087394 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b089a-1239-452c-9cb1-57311fbc995b None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:49.087664 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b089a-1239-452c-9cb1-57311fbc995b None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:49.087927 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b089a-1239-452c-9cb1-57311fbc995b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:49.106361 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:49.126630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab51763-310e-4d6b-af05-baac71ea9b40 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:49.171254 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824cb6f5-2c31-4e39-9022-f366cda83ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:49.172069 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cb6f5-2c31-4e39-9022-f366cda83ead None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:49.172583 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cb6f5-2c31-4e39-9022-f366cda83ead None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:49.172952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cb6f5-2c31-4e39-9022-f366cda83ead None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:49.201657 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976b089a-1239-452c-9cb1-57311fbc995b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:49.203990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976b089a-1239-452c-9cb1-57311fbc995b None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:49.207664 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:49.273067 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824cb6f5-2c31-4e39-9022-f366cda83ead None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:49.275236 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824cb6f5-2c31-4e39-9022-f366cda83ead None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:49.348295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 422/851] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:59:48 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 14:59:49.350484 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf92137-1cf1-4e9f-a3a6-fee7c23d1a40 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:49.350936 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf92137-1cf1-4e9f-a3a6-fee7c23d1a40 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:49.351392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf92137-1cf1-4e9f-a3a6-fee7c23d1a40 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:49.357198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 423/852] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:49.357198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf92137-1cf1-4e9f-a3a6-fee7c23d1a40 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:49.403217 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 430/853] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:59:48 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 14:59:49.404252 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824cb6f5-2c31-4e39-9022-f366cda83ead None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:49.425506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:49.463567 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:49.487890 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf92137-1cf1-4e9f-a3a6-fee7c23d1a40 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:49.508250 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824cb6f5-2c31-4e39-9022-f366cda83ead None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:49.522070 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782fbe1f-ff35-4de2-831a-0d16a808b1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:49.522734 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fbe1f-ff35-4de2-831a-0d16a808b1ad None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:49.523004 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fbe1f-ff35-4de2-831a-0d16a808b1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:49.523272 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fbe1f-ff35-4de2-831a-0d16a808b1ad None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:49.523860 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782fbe1f-ff35-4de2-831a-0d16a808b1ad None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:49.525271 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782fbe1f-ff35-4de2-831a-0d16a808b1ad None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:49.534326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-782fbe1f-ff35-4de2-831a-0d16a808b1ad None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 14:59:49.534326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:49.534326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:59:49.534326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:59:49.534326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:59:49.534326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:59:49.534326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:59:49.534326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:59:49.534326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 14:59:49.534326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:49.534326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:59:49.534326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:49.534326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:49.534326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:49.534326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:49.534326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:49.534326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:49.544230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:49.544230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:49.544230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:49.544230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:49.544230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:49.544230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:49.544230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:59:49.544230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:59:49.544230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:59:49.544230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:59:49.544230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:49.544230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:49.544230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:49.544230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:49.544230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:49.544230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:49.545287 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:49.545287 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:49.545287 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:49.545287 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:49.545287 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:49.545287 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:49.545287 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:49.545287 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:49.545287 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:49.545287 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:49.545287 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:59:49.545287 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:59:49.545287 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 14:59:49.545287 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:49.545287 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 424/854] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 14:59:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:49.546507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e87a0-dc81-4f69-a442-aba99de0a326 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:49.546507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e87a0-dc81-4f69-a442-aba99de0a326 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:49.546507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e87a0-dc81-4f69-a442-aba99de0a326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:49.550383 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca54cd6-f718-4d0c-89e7-63012e8010b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:49.551153 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca54cd6-f718-4d0c-89e7-63012e8010b4 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:49.551528 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca54cd6-f718-4d0c-89e7-63012e8010b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:49.551898 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca54cd6-f718-4d0c-89e7-63012e8010b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:49.555830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-cf6e87a0-dc81-4f69-a442-aba99de0a326 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:49.557042 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-cf6e87a0-dc81-4f69-a442-aba99de0a326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:49.563590 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca54cd6-f718-4d0c-89e7-63012e8010b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:49.565851 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca54cd6-f718-4d0c-89e7-63012e8010b4 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:49.581008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-0ca54cd6-f718-4d0c-89e7-63012e8010b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:49.581008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:49.581008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:49.581008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:49.581008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:49.581008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:49.581008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:49.581008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:49.581008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:49.581008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:49.581008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:49.581008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:49.581008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:49.581008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:49.581008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:49.581008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:49.582499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:49.582499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:49.582499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:49.582499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:49.582499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:49.582499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:49.582499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:49.582499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:49.582499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:49.582499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:49.582499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:49.582499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:49.582499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:49.582499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:49.582499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:49.582499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:49.582499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:49.583511 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:49.583511 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:49.583511 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:49.583511 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 431/855] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:49.584901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca54cd6-f718-4d0c-89e7-63012e8010b4 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:49.634304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 425/856] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:49 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:59:49.634991 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e87a0-dc81-4f69-a442-aba99de0a326 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:49.681951 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:49.723429 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca54cd6-f718-4d0c-89e7-63012e8010b4 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:49.738925 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:49.776607 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c573a1-f477-4c1e-a3b2-be5d27b6571a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:49.777619 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c573a1-f477-4c1e-a3b2-be5d27b6571a None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:49.778174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c573a1-f477-4c1e-a3b2-be5d27b6571a None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:49.778409 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c573a1-f477-4c1e-a3b2-be5d27b6571a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:49.783777 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e87a0-dc81-4f69-a442-aba99de0a326 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:49.795781 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c573a1-f477-4c1e-a3b2-be5d27b6571a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:49.798289 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c573a1-f477-4c1e-a3b2-be5d27b6571a None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:49.821773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-b2c573a1-f477-4c1e-a3b2-be5d27b6571a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:49.821773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:49.821773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:49.821773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:49.821773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:49.821773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:49.821773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:49.821773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:49.821773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:49.821773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:49.821773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:49.821773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:49.821773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:49.821773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:49.821773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:49.821773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:49.822901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:49.822901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:49.822901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:49.822901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:49.822901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:49.822901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:49.822901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:49.822901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:49.822901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:49.822901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:49.822901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:49.822901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:49.822901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:49.822901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:49.822901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:49.822901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:49.822901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:49.823871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:49.823871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:49.823871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:49.829706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 432/857] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:49.831061 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c573a1-f477-4c1e-a3b2-be5d27b6571a None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:49.836869 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f51071-369d-48b2-959b-0356ce0bb64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:49.837683 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f51071-369d-48b2-959b-0356ce0bb64c None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:49.838074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f51071-369d-48b2-959b-0356ce0bb64c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:49.838459 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f51071-369d-48b2-959b-0356ce0bb64c None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:49.839254 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f51071-369d-48b2-959b-0356ce0bb64c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:49.839674 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f51071-369d-48b2-959b-0356ce0bb64c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:49.841285 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f51071-369d-48b2-959b-0356ce0bb64c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:49.852399 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 426/858] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 14:59:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:49.852399 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f51071-369d-48b2-959b-0356ce0bb64c None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:49.954442 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c573a1-f477-4c1e-a3b2-be5d27b6571a None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:49.957629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:49.961806 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f51071-369d-48b2-959b-0356ce0bb64c None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:49.991460 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43591896-f438-4fe1-94a9-ebbb676ab6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:49.991969 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43591896-f438-4fe1-94a9-ebbb676ab6fd None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:49.992243 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43591896-f438-4fe1-94a9-ebbb676ab6fd None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:49.992681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43591896-f438-4fe1-94a9-ebbb676ab6fd None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:49.993360 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43591896-f438-4fe1-94a9-ebbb676ab6fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:49.993665 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43591896-f438-4fe1-94a9-ebbb676ab6fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:49.995024 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43591896-f438-4fe1-94a9-ebbb676ab6fd None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:50.001519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d6336-ad23-41ea-bf7a-a95b5d3f6822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:50.002279 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d6336-ad23-41ea-bf7a-a95b5d3f6822 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:50.002629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d6336-ad23-41ea-bf7a-a95b5d3f6822 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:50.002977 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d6336-ad23-41ea-bf7a-a95b5d3f6822 None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:50.003718 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d6336-ad23-41ea-bf7a-a95b5d3f6822 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:50.004091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d6336-ad23-41ea-bf7a-a95b5d3f6822 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:50.005571 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d6336-ad23-41ea-bf7a-a95b5d3f6822 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:50.007521 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 433/859] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:50.008350 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43591896-f438-4fe1-94a9-ebbb676ab6fd None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:50.023923 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 427/860] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:50.025009 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d6336-ad23-41ea-bf7a-a95b5d3f6822 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:50.031393 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:50.113522 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43591896-f438-4fe1-94a9-ebbb676ab6fd None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:50.131134 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d6336-ad23-41ea-bf7a-a95b5d3f6822 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:50.151205 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5caec4d-a774-4aeb-9ecd-9911bfc9722e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:50.152091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5caec4d-a774-4aeb-9ecd-9911bfc9722e None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:50.152468 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5caec4d-a774-4aeb-9ecd-9911bfc9722e None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:50.152914 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5caec4d-a774-4aeb-9ecd-9911bfc9722e None admin] PATH_INFO: `/v3/services` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:50.153849 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5caec4d-a774-4aeb-9ecd-9911bfc9722e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:50.154202 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5caec4d-a774-4aeb-9ecd-9911bfc9722e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:50.155865 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5caec4d-a774-4aeb-9ecd-9911bfc9722e None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:50.161583 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 434/861] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 14:59:50 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:50.162670 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5caec4d-a774-4aeb-9ecd-9911bfc9722e None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:50.168541 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02e2b7-584b-4655-ae53-5be5b0293a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:50.169245 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02e2b7-584b-4655-ae53-5be5b0293a5d None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:50.169523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02e2b7-584b-4655-ae53-5be5b0293a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:50.169788 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02e2b7-584b-4655-ae53-5be5b0293a5d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:50.170399 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02e2b7-584b-4655-ae53-5be5b0293a5d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:50.171858 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02e2b7-584b-4655-ae53-5be5b0293a5d None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:50.182992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-8d02e2b7-584b-4655-ae53-5be5b0293a5d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 14:59:50.182992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:50.182992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:59:50.182992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:59:50.182992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:59:50.182992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:59:50.182992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:59:50.182992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:59:50.182992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 14:59:50.182992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:50.182992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:59:50.182992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:50.182992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:50.182992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:50.182992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:50.182992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:50.184975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:50.184975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:50.184975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:50.184975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:50.184975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:50.184975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:50.184975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:50.184975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:59:50.184975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:59:50.184975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:59:50.184975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:59:50.184975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:50.184975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:50.184975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:50.184975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:50.184975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:50.184975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:50.186896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:50.186896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:50.186896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:50.186896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:50.186896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:50.186896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:50.186896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:50.186896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:50.186896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:50.186896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:50.186896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:59:50.186896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:59:50.186896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 14:59:50.186896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:50.186896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 428/862] 10.222.0.44 () {56 vars in 1132 bytes} [Sun Sep 11 14:59:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:50.188408 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02e2b7-584b-4655-ae53-5be5b0293a5d None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:50.221588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:50.298120 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:50.319731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5caec4d-a774-4aeb-9ecd-9911bfc9722e None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:50.327163 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02e2b7-584b-4655-ae53-5be5b0293a5d None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:50.367880 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b34e2-6e0b-404e-8bf9-92ddc7d24a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:50.368683 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b34e2-6e0b-404e-8bf9-92ddc7d24a93 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:50.369040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b34e2-6e0b-404e-8bf9-92ddc7d24a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:50.369453 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b34e2-6e0b-404e-8bf9-92ddc7d24a93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:50.377353 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b34e2-6e0b-404e-8bf9-92ddc7d24a93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:50.378810 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b34e2-6e0b-404e-8bf9-92ddc7d24a93 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:50.381183 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d3840-db26-43a3-8b5d-0d0a7df679e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:50.381183 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d3840-db26-43a3-8b5d-0d0a7df679e0 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:50.382083 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d3840-db26-43a3-8b5d-0d0a7df679e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:50.382083 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d3840-db26-43a3-8b5d-0d0a7df679e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:50.392852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-7b6b34e2-6e0b-404e-8bf9-92ddc7d24a93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:50.392852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:50.392852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:50.392852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:50.392852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:50.392852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:50.392852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:50.392852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:50.392852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:50.392852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:50.392852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:50.392852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:50.392852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:59:50.392852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:59:50.392852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:59:50.393839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:59:50.393839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:50.393839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:50.393839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:50.393839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:50.393839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:50.393839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:50.393839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:50.393839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:50.393839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:50.393839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:50.393839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:50.393839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:50.393839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:50.393839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:50.393839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:50.393839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:50.394716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:59:50.394716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:59:50.394716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:59:50.394716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:59:50.394716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:59:50.394716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:59:50.394716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:50.394716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:50.395162 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d3840-db26-43a3-8b5d-0d0a7df679e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:50.395243 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 435/863] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:59:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:50.395243 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b34e2-6e0b-404e-8bf9-92ddc7d24a93 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:50.397285 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d3840-db26-43a3-8b5d-0d0a7df679e0 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:50.419353 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-542d3840-db26-43a3-8b5d-0d0a7df679e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:50.419353 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:50.419353 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:50.419353 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:50.419353 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:50.419353 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:50.419353 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:50.419353 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:50.419353 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:50.419353 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:50.419353 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:50.419353 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:50.419353 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:59:50.419353 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:59:50.419353 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:59:50.421252 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:59:50.421252 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:50.421252 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:50.421252 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:50.421252 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:50.421252 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:50.421252 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:50.421252 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:50.421252 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:50.421252 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:50.421252 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:50.421252 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:50.421252 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:50.421252 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:50.421252 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:50.421252 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:50.421252 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:50.422932 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:59:50.422932 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:59:50.422932 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:59:50.422932 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:59:50.422932 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:59:50.422932 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:59:50.422932 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:50.422932 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:50.424780 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 429/864] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:59:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:50.424780 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d3840-db26-43a3-8b5d-0d0a7df679e0 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:50.557755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:50.574609 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b34e2-6e0b-404e-8bf9-92ddc7d24a93 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:50.588287 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d3840-db26-43a3-8b5d-0d0a7df679e0 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:50.638107 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586eace0-cfdf-45b9-ad00-2ebd31736865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:50.639402 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586eace0-cfdf-45b9-ad00-2ebd31736865 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:50.639878 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586eace0-cfdf-45b9-ad00-2ebd31736865 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:50.640303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586eace0-cfdf-45b9-ad00-2ebd31736865 None admin] PATH_INFO: `/v3/services` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:50.640845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6d17f-646b-4841-aa00-c5bad7d39be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:50.641548 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586eace0-cfdf-45b9-ad00-2ebd31736865 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:50.641710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6d17f-646b-4841-aa00-c5bad7d39be7 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:50.641848 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6d17f-646b-4841-aa00-c5bad7d39be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:50.642155 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6d17f-646b-4841-aa00-c5bad7d39be7 None admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:50.642326 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586eace0-cfdf-45b9-ad00-2ebd31736865 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:50.642751 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b6d17f-646b-4841-aa00-c5bad7d39be7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:50.643054 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b6d17f-646b-4841-aa00-c5bad7d39be7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:50.644478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b6d17f-646b-4841-aa00-c5bad7d39be7 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:50.644658 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586eace0-cfdf-45b9-ad00-2ebd31736865 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:50.661122 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 436/865] 10.222.0.44 () {56 vars in 1126 bytes} [Sun Sep 11 14:59:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:50.661122 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586eace0-cfdf-45b9-ad00-2ebd31736865 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:50.666662 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 430/866] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 14:59:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:50.670468 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6d17f-646b-4841-aa00-c5bad7d39be7 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:50.673992 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:50.776978 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586eace0-cfdf-45b9-ad00-2ebd31736865 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:50.817090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee98412e-7a2c-4fe3-9c4d-405664398279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:50.817735 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98412e-7a2c-4fe3-9c4d-405664398279 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:50.818027 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98412e-7a2c-4fe3-9c4d-405664398279 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:50.818287 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98412e-7a2c-4fe3-9c4d-405664398279 None admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:50.819085 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee98412e-7a2c-4fe3-9c4d-405664398279 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:50.819389 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee98412e-7a2c-4fe3-9c4d-405664398279 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:50.820832 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee98412e-7a2c-4fe3-9c4d-405664398279 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:50.829246 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6d17f-646b-4841-aa00-c5bad7d39be7 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:50.840393 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 437/867] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 14:59:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:50.840393 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee98412e-7a2c-4fe3-9c4d-405664398279 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:50.887963 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5de4a-5b95-43c9-97d3-ae9003426636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:50.888961 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5de4a-5b95-43c9-97d3-ae9003426636 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:50.889466 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5de4a-5b95-43c9-97d3-ae9003426636 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:50.889914 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5de4a-5b95-43c9-97d3-ae9003426636 None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:50.891011 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d5de4a-5b95-43c9-97d3-ae9003426636 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:50.891528 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d5de4a-5b95-43c9-97d3-ae9003426636 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:50.894061 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d5de4a-5b95-43c9-97d3-ae9003426636 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:50.909167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 431/868] 10.222.0.44 () {56 vars in 1126 bytes} [Sun Sep 11 14:59:50 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:50.909167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5de4a-5b95-43c9-97d3-ae9003426636 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:50.920234 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:50.927740 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:50.994755 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee98412e-7a2c-4fe3-9c4d-405664398279 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:51.044890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb8adf-b04e-474a-a66d-4adac098fb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:51.045959 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb8adf-b04e-474a-a66d-4adac098fb75 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:51.045959 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb8adf-b04e-474a-a66d-4adac098fb75 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:51.046350 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb8adf-b04e-474a-a66d-4adac098fb75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:51.047342 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb8adf-b04e-474a-a66d-4adac098fb75 None admin] RBAC: Adding query filter params (role.id=ea3b2f1eb283418b8823cb4fff80edba, scope.project.id=70ac515938a24c45ae6302dbac46a1ab, user.id=3fada78ceaeb4f3bad77c516f6dfcfd4) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:51.047655 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb8adf-b04e-474a-a66d-4adac098fb75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:51.049390 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb8adf-b04e-474a-a66d-4adac098fb75 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:51.061230 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 438/869] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:59:50 2022] GET /identity/v3/role_assignments?user.id=3fada78ceaeb4f3bad77c516f6dfcfd4&scope.project.id=70ac515938a24c45ae6302dbac46a1ab&role.id=ea3b2f1eb283418b8823cb4fff80edba => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:51.085653 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5de4a-5b95-43c9-97d3-ae9003426636 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:51.140056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9610f4b-3eb4-4e68-a67e-4285f63315a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:51.140674 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9610f4b-3eb4-4e68-a67e-4285f63315a6 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:51.140962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9610f4b-3eb4-4e68-a67e-4285f63315a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:51.141262 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9610f4b-3eb4-4e68-a67e-4285f63315a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:51.141998 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9610f4b-3eb4-4e68-a67e-4285f63315a6 None admin] RBAC: Adding query filter params (role.id=c426875a15834aa5b122e2a62267d15a, scope.project.id=70ac515938a24c45ae6302dbac46a1ab, user.id=05aa5e40ee4542719657bc03b2015e72) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:51.142304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9610f4b-3eb4-4e68-a67e-4285f63315a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:51.143826 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9610f4b-3eb4-4e68-a67e-4285f63315a6 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:51.152315 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 432/870] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:59:50 2022] GET /identity/v3/role_assignments?user.id=05aa5e40ee4542719657bc03b2015e72&scope.project.id=70ac515938a24c45ae6302dbac46a1ab&role.id=c426875a15834aa5b122e2a62267d15a => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:52.890949 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4775ca-504d-45d4-b0a4-b18e23a1185c None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:52.891494 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4775ca-504d-45d4-b0a4-b18e23a1185c None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:52.891865 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4775ca-504d-45d4-b0a4-b18e23a1185c None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:52.894709 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 439/871] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:52.903658 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701d986-5f08-43f9-a420-dfcbea5d24f6 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:52.904212 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701d986-5f08-43f9-a420-dfcbea5d24f6 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:52.904567 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701d986-5f08-43f9-a420-dfcbea5d24f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:52.934183 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-2701d986-5f08-43f9-a420-dfcbea5d24f6 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:52.936025 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-2701d986-5f08-43f9-a420-dfcbea5d24f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:53.056654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:53.065803 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 433/872] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:59:53.235634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899d21b-65fc-4f32-b7c2-bf4ce34f8a2c None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:53.236036 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899d21b-65fc-4f32-b7c2-bf4ce34f8a2c None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:53.236522 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899d21b-65fc-4f32-b7c2-bf4ce34f8a2c None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:53.239343 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 440/873] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:53.245552 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e165b8-a5e8-4e27-b308-c166a6b7438d None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:53.246254 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e165b8-a5e8-4e27-b308-c166a6b7438d None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:53.246727 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e165b8-a5e8-4e27-b308-c166a6b7438d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:53.277691 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-f5e165b8-a5e8-4e27-b308-c166a6b7438d None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:53.279124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f5e165b8-a5e8-4e27-b308-c166a6b7438d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:53.365312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 434/874] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:59:53.463282 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad70a2-6d18-42ed-b014-b7493eece1fc None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:53.464023 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad70a2-6d18-42ed-b014-b7493eece1fc None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:53.464492 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad70a2-6d18-42ed-b014-b7493eece1fc None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:53.466344 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 441/875] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:53.475323 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4f22e-9ce0-460a-a3d0-57d2db8f4d33 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:53.475910 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4f22e-9ce0-460a-a3d0-57d2db8f4d33 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:53.476376 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4f22e-9ce0-460a-a3d0-57d2db8f4d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:53.496339 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-29b4f22e-9ce0-460a-a3d0-57d2db8f4d33 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:53.497928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-29b4f22e-9ce0-460a-a3d0-57d2db8f4d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:53.577159 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad70a2-6d18-42ed-b014-b7493eece1fc None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:53.577612 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 435/876] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:59:53.646520 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b4f22e-9ce0-460a-a3d0-57d2db8f4d33 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:53.718024 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:53.780387 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad70a2-6d18-42ed-b014-b7493eece1fc None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:53.781360 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b4f22e-9ce0-460a-a3d0-57d2db8f4d33 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:53.822221 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef65d67-3590-42e3-a221-b14314952ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:53.823284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef65d67-3590-42e3-a221-b14314952ff0 None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:53.823755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef65d67-3590-42e3-a221-b14314952ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:53.824238 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef65d67-3590-42e3-a221-b14314952ff0 None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:53.825205 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef65d67-3590-42e3-a221-b14314952ff0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:53.826801 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef65d67-3590-42e3-a221-b14314952ff0 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:53.850491 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 436/877] 10.222.0.44 () {60 vars in 1141 bytes} [Sun Sep 11 14:59:53 2022] POST /identity/v3/services => generated 269 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 14:59:53.851182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:53.859571 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef65d67-3590-42e3-a221-b14314952ff0 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:53.873863 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bae974-04ad-4242-9953-c98c8a43f0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:53.874886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bae974-04ad-4242-9953-c98c8a43f0e6 None admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:53.875372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bae974-04ad-4242-9953-c98c8a43f0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:53.875842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bae974-04ad-4242-9953-c98c8a43f0e6 None admin] PATH_INFO: `/v3/services` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:53.877297 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bae974-04ad-4242-9953-c98c8a43f0e6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:53.880270 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bae974-04ad-4242-9953-c98c8a43f0e6 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:53.920586 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 442/878] 10.222.0.44 () {60 vars in 1141 bytes} [Sun Sep 11 14:59:53 2022] POST /identity/v3/services => generated 251 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:59:53.994963 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef65d67-3590-42e3-a221-b14314952ff0 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:54.035815 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:54.078703 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f46632-40d7-4383-a00a-c170bdcc5c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:54.079681 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f46632-40d7-4383-a00a-c170bdcc5c72 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:54.080177 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f46632-40d7-4383-a00a-c170bdcc5c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:54.080605 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f46632-40d7-4383-a00a-c170bdcc5c72 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:54.081649 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f46632-40d7-4383-a00a-c170bdcc5c72 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:54.084280 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f46632-40d7-4383-a00a-c170bdcc5c72 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:54.102131 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-12f46632-40d7-4383-a00a-c170bdcc5c72 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 14:59:54.102131 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:54.102131 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:59:54.102131 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:59:54.102131 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:59:54.102131 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:59:54.102131 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:59:54.102131 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:59:54.102131 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 14:59:54.102131 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:54.102131 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:59:54.102131 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:54.102131 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:54.102131 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:54.102131 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:54.102131 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:54.102131 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:54.103210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:54.103210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:54.103210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:54.103210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:54.103210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:54.103210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:54.103210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:59:54.103210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:59:54.103210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:59:54.103210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:59:54.103210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:54.103210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:54.103210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:54.103210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:54.103210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:54.103210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:54.104040 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:54.104040 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:54.104040 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:54.104040 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:54.104040 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:54.104040 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:54.104040 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:54.104040 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:54.104040 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:54.104040 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:54.104040 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:59:54.104040 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:59:54.104040 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 14:59:54.104040 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:54.105488 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 437/879] 10.222.0.44 () {56 vars in 1126 bytes} [Sun Sep 11 14:59:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:59:54.112819 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bae974-04ad-4242-9953-c98c8a43f0e6 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:54.302902 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bae974-04ad-4242-9953-c98c8a43f0e6 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:54.368031 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914aac79-258b-4fc3-905b-8258454f3c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:54.369025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914aac79-258b-4fc3-905b-8258454f3c30 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:54.369469 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914aac79-258b-4fc3-905b-8258454f3c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:54.369864 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914aac79-258b-4fc3-905b-8258454f3c30 None admin] PATH_INFO: `/v3/services` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:54.370930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914aac79-258b-4fc3-905b-8258454f3c30 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:54.371384 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914aac79-258b-4fc3-905b-8258454f3c30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:54.373670 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914aac79-258b-4fc3-905b-8258454f3c30 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:54.382414 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 443/880] 10.222.0.44 () {56 vars in 1122 bytes} [Sun Sep 11 14:59:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:54.388632 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f46632-40d7-4383-a00a-c170bdcc5c72 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:54.558535 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f46632-40d7-4383-a00a-c170bdcc5c72 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:54.561529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1aa1c2-bea2-4150-8516-45dd50852bd0 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:54.561806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1aa1c2-bea2-4150-8516-45dd50852bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:54.562078 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1aa1c2-bea2-4150-8516-45dd50852bd0 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:54.563664 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 444/881] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:54.569090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445865b0-5a2d-40bb-a174-c642ec2b2673 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:54.569422 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445865b0-5a2d-40bb-a174-c642ec2b2673 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:54.569654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445865b0-5a2d-40bb-a174-c642ec2b2673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:54.588405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-445865b0-5a2d-40bb-a174-c642ec2b2673 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:54.589786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-445865b0-5a2d-40bb-a174-c642ec2b2673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:54.630117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e16439e-afe5-472e-a40a-d7e81645faea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:54.631722 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16439e-afe5-472e-a40a-d7e81645faea None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:54.632455 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16439e-afe5-472e-a40a-d7e81645faea None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:54.633179 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16439e-afe5-472e-a40a-d7e81645faea None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:54.634643 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e16439e-afe5-472e-a40a-d7e81645faea None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:54.635452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e16439e-afe5-472e-a40a-d7e81645faea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:54.638755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e16439e-afe5-472e-a40a-d7e81645faea None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:54.642125 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:54.650497 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 438/882] 10.222.0.44 () {56 vars in 1122 bytes} [Sun Sep 11 14:59:54 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:54.682738 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 445/883] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:54 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 14:59:55.045565 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e16439e-afe5-472e-a40a-d7e81645faea None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:55.130244 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:55.232049 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e16439e-afe5-472e-a40a-d7e81645faea None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:55.296320 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc392df-af96-4432-b607-88abcab2489c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:55.297241 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc392df-af96-4432-b607-88abcab2489c None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:55.297707 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc392df-af96-4432-b607-88abcab2489c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:55.298167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc392df-af96-4432-b607-88abcab2489c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:55.317062 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc392df-af96-4432-b607-88abcab2489c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 14:59:55.326347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-dcc392df-af96-4432-b607-88abcab2489c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 14:59:55.326347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:55.326347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:55.326347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:55.326347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:55.326347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:55.326347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:55.326347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:55.326347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:55.326347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:55.326347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:55.326347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:55.326347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 14:59:55.326347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 14:59:55.326347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 14:59:55.326347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 14:59:55.328117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 14:59:55.328117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 14:59:55.328117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 14:59:55.328117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 14:59:55.328117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:55.328117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:55.328117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:55.328117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:55.328117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:55.328117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:55.328117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:55.328117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:55.328117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:55.328117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:55.328117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:55.328117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:55.329373 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:55.329373 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:55.329373 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:55.329373 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:55.329373 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 14:59:55.329373 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 14:59:55.329373 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 14:59:55.329373 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 14:59:55.329373 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 14:59:55.329373 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 14:59:55.329373 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 14:59:55.329373 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:55.329373 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 439/884] 10.222.0.44 () {56 vars in 1120 bytes} [Sun Sep 11 14:59:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 14:59:55.342186 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445865b0-5a2d-40bb-a174-c642ec2b2673 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:55.452821 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445865b0-5a2d-40bb-a174-c642ec2b2673 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:55.490071 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a25f67-a637-4308-8585-8d604503f78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:55.490751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a25f67-a637-4308-8585-8d604503f78b None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:55.491023 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a25f67-a637-4308-8585-8d604503f78b None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:55.491285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a25f67-a637-4308-8585-8d604503f78b None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:55.491968 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a25f67-a637-4308-8585-8d604503f78b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:55.492288 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a25f67-a637-4308-8585-8d604503f78b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:55.493723 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a25f67-a637-4308-8585-8d604503f78b None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:55.503392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 446/885] 10.222.0.44 () {56 vars in 1115 bytes} [Sun Sep 11 14:59:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:55.512619 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc392df-af96-4432-b607-88abcab2489c None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:55.691057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc392df-af96-4432-b607-88abcab2489c None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:55.695734 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:55.754965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8bbcf9-21e5-406c-afc0-0e46b58b0957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:55.756009 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bbcf9-21e5-406c-afc0-0e46b58b0957 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:55.756479 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bbcf9-21e5-406c-afc0-0e46b58b0957 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:55.756939 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bbcf9-21e5-406c-afc0-0e46b58b0957 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:55.776432 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8bbcf9-21e5-406c-afc0-0e46b58b0957 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:55.779128 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8bbcf9-21e5-406c-afc0-0e46b58b0957 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:55.807962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-6a8bbcf9-21e5-406c-afc0-0e46b58b0957 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:55.807962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:55.807962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:55.807962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:55.807962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:55.807962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:55.807962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:55.807962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:55.807962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:55.807962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:55.807962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:55.807962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:55.807962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:55.807962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:55.807962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:55.807962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:55.809921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:55.809921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:55.809921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:55.809921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:55.809921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:55.809921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:55.809921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:55.809921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:55.809921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:55.809921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:55.809921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:55.809921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:55.809921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:55.809921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:55.809921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:55.809921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:55.809921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:55.811585 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:55.811585 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:55.811585 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:55.811585 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 440/886] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:55.819319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a25f67-a637-4308-8585-8d604503f78b None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:55.907043 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:56.018427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a25f67-a637-4308-8585-8d604503f78b None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:56.084794 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db048d4c-a4d2-42e8-a281-352fc0161645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:56.085940 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db048d4c-a4d2-42e8-a281-352fc0161645 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:56.086412 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db048d4c-a4d2-42e8-a281-352fc0161645 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:56.086890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db048d4c-a4d2-42e8-a281-352fc0161645 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:56.088062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db048d4c-a4d2-42e8-a281-352fc0161645 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:56.088610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db048d4c-a4d2-42e8-a281-352fc0161645 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:56.091333 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db048d4c-a4d2-42e8-a281-352fc0161645 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:56.115646 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 447/887] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:56.123343 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8bbcf9-21e5-406c-afc0-0e46b58b0957 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:56.321805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8bbcf9-21e5-406c-afc0-0e46b58b0957 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:56.390067 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b0e08-ea06-4bf6-9b10-592c8d3cdb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:56.391115 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b0e08-ea06-4bf6-9b10-592c8d3cdb7a None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:56.391568 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b0e08-ea06-4bf6-9b10-592c8d3cdb7a None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:56.392003 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b0e08-ea06-4bf6-9b10-592c8d3cdb7a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:56.406776 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65b0e08-ea06-4bf6-9b10-592c8d3cdb7a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:56.409502 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65b0e08-ea06-4bf6-9b10-592c8d3cdb7a None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:56.432176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-d65b0e08-ea06-4bf6-9b10-592c8d3cdb7a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 14:59:56.432176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:56.432176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:56.432176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:56.432176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:56.432176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:56.432176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:56.432176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:56.432176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:56.432176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:56.432176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:56.432176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:56.432176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 14:59:56.432176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 14:59:56.432176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 14:59:56.432176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 14:59:56.433537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:56.433537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:56.433537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 14:59:56.433537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:56.433537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 14:59:56.433537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 14:59:56.433537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:56.433537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:56.433537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:56.433537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:56.433537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:56.433537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:56.433537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:56.433537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:56.433537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:56.433537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:56.433537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:56.433537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:56.434478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:56.434478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:56.434478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 14:59:56.434478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 14:59:56.434478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 14:59:56.434478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 14:59:56.434478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 14:59:56.434478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 14:59:56.434478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 14:59:56.434478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:56.435215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 441/888] 10.222.0.44 () {56 vars in 1117 bytes} [Sun Sep 11 14:59:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:56.442024 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db048d4c-a4d2-42e8-a281-352fc0161645 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:56.489777 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3078e-4459-48c8-863b-e7800f052838 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:56.490664 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3078e-4459-48c8-863b-e7800f052838 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:56.491506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3078e-4459-48c8-863b-e7800f052838 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:56.494722 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 442/889] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:56.504538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359c288-c8d8-4363-a810-56ddf11a2cab None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:56.505522 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359c288-c8d8-4363-a810-56ddf11a2cab None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:56.506377 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359c288-c8d8-4363-a810-56ddf11a2cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:56.539000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-e359c288-c8d8-4363-a810-56ddf11a2cab None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:56.542171 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-e359c288-c8d8-4363-a810-56ddf11a2cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:56.582116 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db048d4c-a4d2-42e8-a281-352fc0161645 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:56.588730 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:56.625562 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:56.660749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ab48b-4d54-4f26-8531-38e42761b95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:56.662941 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ab48b-4d54-4f26-8531-38e42761b95e None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:56.664086 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ab48b-4d54-4f26-8531-38e42761b95e None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:56.664822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ab48b-4d54-4f26-8531-38e42761b95e None admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:56.666482 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ab48b-4d54-4f26-8531-38e42761b95e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:56.667424 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ab48b-4d54-4f26-8531-38e42761b95e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:56.677603 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 443/890] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:56 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:59:56.677916 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ab48b-4d54-4f26-8531-38e42761b95e None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:56.716200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 448/891] 10.222.0.44 () {56 vars in 1149 bytes} [Sun Sep 11 14:59:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:56.726352 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e359c288-c8d8-4363-a810-56ddf11a2cab None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:56.920162 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbe354-3498-4072-ae2b-c4836a2cf093 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:56.921821 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbe354-3498-4072-ae2b-c4836a2cf093 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:56.922748 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbe354-3498-4072-ae2b-c4836a2cf093 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:56.927610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 449/892] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:56 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 14:59:56.931119 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677d81a-89ac-495d-8800-719733f77b34 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:56.931435 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677d81a-89ac-495d-8800-719733f77b34 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:56.931735 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677d81a-89ac-495d-8800-719733f77b34 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:56.933158 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 450/893] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 14:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 14:59:56.934333 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2dbc97-1892-4844-a925-4d1329b01171 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:56.934612 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2dbc97-1892-4844-a925-4d1329b01171 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:56.934878 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2dbc97-1892-4844-a925-4d1329b01171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:56.935207 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e359c288-c8d8-4363-a810-56ddf11a2cab None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:56.955397 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-9b2dbc97-1892-4844-a925-4d1329b01171 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:56.956714 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-9b2dbc97-1892-4844-a925-4d1329b01171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:56.998023 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368649d-ee7c-4bc7-9262-fabf0f0a0ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:56.999019 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368649d-ee7c-4bc7-9262-fabf0f0a0ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:56.999466 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368649d-ee7c-4bc7-9262-fabf0f0a0ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:56.999859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368649d-ee7c-4bc7-9262-fabf0f0a0ce6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:57.025928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:57.035362 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 451/894] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:56 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:59:57.039375 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50995199-4d11-4195-bdbb-032577f1ee7d None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:57.039846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50995199-4d11-4195-bdbb-032577f1ee7d None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:57.040255 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50995199-4d11-4195-bdbb-032577f1ee7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:57.072365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-50995199-4d11-4195-bdbb-032577f1ee7d None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 14:59:57.074433 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-50995199-4d11-4195-bdbb-032577f1ee7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 14:59:57.108257 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7368649d-ee7c-4bc7-9262-fabf0f0a0ce6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:57.110181 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7368649d-ee7c-4bc7-9262-fabf0f0a0ce6 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:57.186653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 452/895] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 14:59:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 14:59:57.187957 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50995199-4d11-4195-bdbb-032577f1ee7d None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:57.238641 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:57.277370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 444/896] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 14:59:56 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 14:59:57.286238 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368649d-ee7c-4bc7-9262-fabf0f0a0ce6 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:57.344764 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:57.363816 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50995199-4d11-4195-bdbb-032577f1ee7d None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:57.407827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd729820-88d5-41ad-a8aa-64bf78a97ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:57.408587 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd729820-88d5-41ad-a8aa-64bf78a97ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:57.408866 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd729820-88d5-41ad-a8aa-64bf78a97ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:57.409124 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd729820-88d5-41ad-a8aa-64bf78a97ff0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:57.409806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd729820-88d5-41ad-a8aa-64bf78a97ff0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:57.411213 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd729820-88d5-41ad-a8aa-64bf78a97ff0 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:57.424843 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-fd729820-88d5-41ad-a8aa-64bf78a97ff0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 14:59:57.424843 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:57.424843 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:59:57.424843 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:59:57.424843 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:59:57.424843 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:59:57.424843 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:59:57.424843 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:59:57.424843 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 14:59:57.424843 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:57.424843 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:59:57.424843 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:57.424843 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:57.424843 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:57.424843 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:57.424843 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:57.426392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:57.426392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:57.426392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:57.426392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:57.426392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:57.426392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:57.426392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:57.426392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:59:57.426392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:59:57.426392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:59:57.426392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:59:57.426392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:57.426392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:57.426392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:57.426392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:57.426392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:57.426392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:57.428202 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:57.428202 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:57.428202 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:57.428202 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:57.428202 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:57.428202 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:57.428202 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:57.428202 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:57.428202 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:57.428202 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:57.428202 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:59:57.428202 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:59:57.428202 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 14:59:57.428202 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:57.433537 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 453/897] 10.222.0.44 () {56 vars in 1132 bytes} [Sun Sep 11 14:59:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:57.439904 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd729820-88d5-41ad-a8aa-64bf78a97ff0 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:57.455964 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368649d-ee7c-4bc7-9262-fabf0f0a0ce6 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:57.496075 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fff57-1b77-4e38-8c2d-37803ec97e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:57.497015 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fff57-1b77-4e38-8c2d-37803ec97e1f None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:57.497015 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fff57-1b77-4e38-8c2d-37803ec97e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:57.497357 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fff57-1b77-4e38-8c2d-37803ec97e1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:57.508827 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8fff57-1b77-4e38-8c2d-37803ec97e1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:57.510405 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8fff57-1b77-4e38-8c2d-37803ec97e1f None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:57.528109 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-6f8fff57-1b77-4e38-8c2d-37803ec97e1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:57.528109 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:57.528109 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:57.528109 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:57.528109 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:57.528109 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:57.528109 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:57.528109 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:57.528109 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:57.528109 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:57.528109 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:57.528109 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:57.528109 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 14:59:57.528109 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 14:59:57.528109 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 14:59:57.528109 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 14:59:57.530352 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:57.530352 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:57.530352 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:57.530352 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:57.530352 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:57.530352 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:57.530352 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:57.530352 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:57.530352 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:57.530352 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:57.530352 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:57.530352 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:57.530352 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:57.530352 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:57.530352 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:57.530352 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:57.530352 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 14:59:57.532145 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 14:59:57.532145 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 14:59:57.532145 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 14:59:57.532145 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 445/898] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 14:59:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:57.532145 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fff57-1b77-4e38-8c2d-37803ec97e1f None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:57.607140 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:57.634705 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd729820-88d5-41ad-a8aa-64bf78a97ff0 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:57.637839 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:57.642021 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fff57-1b77-4e38-8c2d-37803ec97e1f None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:57.681270 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa557a-8640-4683-80be-1e7c3c50c355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:57.681901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa557a-8640-4683-80be-1e7c3c50c355 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:57.682176 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa557a-8640-4683-80be-1e7c3c50c355 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:57.682521 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa557a-8640-4683-80be-1e7c3c50c355 None admin] PATH_INFO: `/v3/services` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:57.683191 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa557a-8640-4683-80be-1e7c3c50c355 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:57.683495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa557a-8640-4683-80be-1e7c3c50c355 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:57.684921 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa557a-8640-4683-80be-1e7c3c50c355 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:57.691875 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 454/899] 10.222.0.44 () {56 vars in 1126 bytes} [Sun Sep 11 14:59:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 14:59:57.692659 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa557a-8640-4683-80be-1e7c3c50c355 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:57.697833 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38377318-1136-466c-b99b-04dca059d362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:57.698918 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38377318-1136-466c-b99b-04dca059d362 None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:57.699386 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38377318-1136-466c-b99b-04dca059d362 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:57.699792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38377318-1136-466c-b99b-04dca059d362 None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:57.700877 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38377318-1136-466c-b99b-04dca059d362 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:57.703172 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38377318-1136-466c-b99b-04dca059d362 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:57.725826 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 446/900] 10.222.0.44 () {60 vars in 1141 bytes} [Sun Sep 11 14:59:57 2022] POST /identity/v3/services => generated 241 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 14:59:57.727366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38377318-1136-466c-b99b-04dca059d362 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:57.835258 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa557a-8640-4683-80be-1e7c3c50c355 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:57.890552 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a663a2-570d-46ae-85e6-5faccf098dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:57.891209 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a663a2-570d-46ae-85e6-5faccf098dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:57.891482 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a663a2-570d-46ae-85e6-5faccf098dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:57.891748 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a663a2-570d-46ae-85e6-5faccf098dd8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:57.892352 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a663a2-570d-46ae-85e6-5faccf098dd8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:57.893837 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a663a2-570d-46ae-85e6-5faccf098dd8 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:57.904445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-07a663a2-570d-46ae-85e6-5faccf098dd8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 14:59:57.904445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:57.904445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 14:59:57.904445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 14:59:57.904445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 14:59:57.904445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 14:59:57.904445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 14:59:57.904445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:59:57.904445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 14:59:57.904445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:57.904445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 14:59:57.904445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:57.904445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:57.904445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:57.904445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:57.904445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:57.904445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:57.906311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:57.906311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:57.906311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:57.906311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:57.906311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:57.906311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:57.906311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 14:59:57.906311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 14:59:57.906311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 14:59:57.906311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 14:59:57.906311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:57.906311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:57.906311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:57.906311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:57.906311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:57.906311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:57.907886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:57.907886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:57.907886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:57.907886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:57.907886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:57.907886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:57.907886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:57.907886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:57.907886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:57.907886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:57.907886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 14:59:57.907886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 14:59:57.907886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 14:59:57.907886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:57.907886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 455/901] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 14:59:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:57.909456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a663a2-570d-46ae-85e6-5faccf098dd8 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:57.922470 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38377318-1136-466c-b99b-04dca059d362 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:57.931073 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:57.967485 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcba726-15de-4299-9641-0e088780c98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:57.968015 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcba726-15de-4299-9641-0e088780c98e None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:57.968294 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcba726-15de-4299-9641-0e088780c98e None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:57.968562 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcba726-15de-4299-9641-0e088780c98e None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:57.969257 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcba726-15de-4299-9641-0e088780c98e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:57.969560 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcba726-15de-4299-9641-0e088780c98e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:57.971351 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcba726-15de-4299-9641-0e088780c98e None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:57.985641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 447/902] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 14:59:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:57.986537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcba726-15de-4299-9641-0e088780c98e None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:58.013104 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:58.091600 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a663a2-570d-46ae-85e6-5faccf098dd8 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:58.136605 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e776e-0b38-40b3-92e8-2435af1ea31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:58.137611 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e776e-0b38-40b3-92e8-2435af1ea31b None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:58.138618 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e776e-0b38-40b3-92e8-2435af1ea31b None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:58.139087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e776e-0b38-40b3-92e8-2435af1ea31b None admin] PATH_INFO: `/v3/services` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:58.139905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e776e-0b38-40b3-92e8-2435af1ea31b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:58.140540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e776e-0b38-40b3-92e8-2435af1ea31b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:58.142269 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e776e-0b38-40b3-92e8-2435af1ea31b None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:58.154037 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 456/903] 10.222.0.44 () {56 vars in 1126 bytes} [Sun Sep 11 14:59:57 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:58.155620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e776e-0b38-40b3-92e8-2435af1ea31b None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:58.164440 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcba726-15de-4299-9641-0e088780c98e None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:58.219904 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9acb8-ded9-471d-a057-b5e9b5803f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:58.221195 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9acb8-ded9-471d-a057-b5e9b5803f32 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:58.221668 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9acb8-ded9-471d-a057-b5e9b5803f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:58.222131 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9acb8-ded9-471d-a057-b5e9b5803f32 None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:58.223060 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a9acb8-ded9-471d-a057-b5e9b5803f32 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:58.223586 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a9acb8-ded9-471d-a057-b5e9b5803f32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:58.226206 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a9acb8-ded9-471d-a057-b5e9b5803f32 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:58.242212 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 448/904] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 14:59:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:58.243716 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9acb8-ded9-471d-a057-b5e9b5803f32 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:58.290380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:58.337467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e776e-0b38-40b3-92e8-2435af1ea31b None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:58.381722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a820526-d1a9-43ee-9478-89d0e13b3632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:58.382563 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a820526-d1a9-43ee-9478-89d0e13b3632 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:58.382958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a820526-d1a9-43ee-9478-89d0e13b3632 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:58.383575 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a820526-d1a9-43ee-9478-89d0e13b3632 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:58.393486 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a820526-d1a9-43ee-9478-89d0e13b3632 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:58.395567 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a820526-d1a9-43ee-9478-89d0e13b3632 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:58.413419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:58.414484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-9a820526-d1a9-43ee-9478-89d0e13b3632 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:58.414484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 14:59:58.414484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 14:59:58.414484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 14:59:58.414484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 14:59:58.414484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 14:59:58.414484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 14:59:58.414484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 14:59:58.414484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 14:59:58.414484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 14:59:58.414484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 14:59:58.414484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 14:59:58.414484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 14:59:58.414484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 14:59:58.414484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 14:59:58.415346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 14:59:58.415346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 14:59:58.415346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 14:59:58.415346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 14:59:58.415346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 14:59:58.415346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 14:59:58.415346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 14:59:58.415346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 14:59:58.415346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 14:59:58.415346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 14:59:58.415346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 14:59:58.415346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 14:59:58.415346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 14:59:58.415346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 14:59:58.415346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 14:59:58.415346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 14:59:58.415346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 14:59:58.416226 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 14:59:58.416226 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 14:59:58.416226 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 14:59:58.416226 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 14:59:58.416226 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 14:59:58.416226 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 14:59:58.416226 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 14:59:58.416226 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 14:59:58.421295 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 457/905] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 14:59:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 14:59:58.425788 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a820526-d1a9-43ee-9478-89d0e13b3632 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:58.442477 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9acb8-ded9-471d-a057-b5e9b5803f32 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:58.487623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215363b-52a5-479f-9af5-120fa812fb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:58.488467 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215363b-52a5-479f-9af5-120fa812fb20 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:58.488891 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215363b-52a5-479f-9af5-120fa812fb20 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:58.489325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215363b-52a5-479f-9af5-120fa812fb20 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:58.490174 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215363b-52a5-479f-9af5-120fa812fb20 None admin] RBAC: Adding query filter params (interface=public, service_id=dd2ebf6ee9fe4ae9ad245512b39db5e1, region_id=RegionOne) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:58.490614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215363b-52a5-479f-9af5-120fa812fb20 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:58.492610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215363b-52a5-479f-9af5-120fa812fb20 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:58.506273 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 449/906] 10.222.0.44 () {56 vars in 1251 bytes} [Sun Sep 11 14:59:58 2022] GET /identity/v3/endpoints?service_id=dd2ebf6ee9fe4ae9ad245512b39db5e1&interface=public®ion_id=RegionOne => generated 194 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:58.506273 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215363b-52a5-479f-9af5-120fa812fb20 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:58.555278 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a820526-d1a9-43ee-9478-89d0e13b3632 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:58.596172 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80f8ec-5f51-4880-8729-dcfc51462846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:58.596799 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80f8ec-5f51-4880-8729-dcfc51462846 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:58.597073 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80f8ec-5f51-4880-8729-dcfc51462846 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:58.597375 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80f8ec-5f51-4880-8729-dcfc51462846 None admin] PATH_INFO: `/v3/services` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:58.598038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b80f8ec-5f51-4880-8729-dcfc51462846 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:58.598343 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b80f8ec-5f51-4880-8729-dcfc51462846 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:58.600013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b80f8ec-5f51-4880-8729-dcfc51462846 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:58.602256 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:58.609334 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 458/907] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 14:59:58 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:58.610341 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80f8ec-5f51-4880-8729-dcfc51462846 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:58.686095 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215363b-52a5-479f-9af5-120fa812fb20 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:58.731383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7906a-08b9-4944-9172-d44498aff4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:58.732010 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7906a-08b9-4944-9172-d44498aff4c7 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:58.732272 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7906a-08b9-4944-9172-d44498aff4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:58.732544 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7906a-08b9-4944-9172-d44498aff4c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:58.733618 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7906a-08b9-4944-9172-d44498aff4c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:58.733914 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7906a-08b9-4944-9172-d44498aff4c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:58.736015 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7906a-08b9-4944-9172-d44498aff4c7 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:58.760630 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 450/908] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 14:59:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:58.760630 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7906a-08b9-4944-9172-d44498aff4c7 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:58.767177 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80f8ec-5f51-4880-8729-dcfc51462846 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:58.771800 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:58.824726 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7866690-565b-4261-835a-78c4540ba5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:58.825723 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7866690-565b-4261-835a-78c4540ba5b2 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:58.826152 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7866690-565b-4261-835a-78c4540ba5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:58.826518 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7866690-565b-4261-835a-78c4540ba5b2 None admin] PATH_INFO: `/v3/services` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:58.827437 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7866690-565b-4261-835a-78c4540ba5b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:58.829585 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7866690-565b-4261-835a-78c4540ba5b2 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:58.841242 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 459/909] 10.222.0.44 () {56 vars in 1087 bytes} [Sun Sep 11 14:59:58 2022] GET /identity/v3/services => generated 1497 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 14:59:58.842683 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7866690-565b-4261-835a-78c4540ba5b2 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:58.970486 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:58.979757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7906a-08b9-4944-9172-d44498aff4c7 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:59.049727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7866690-565b-4261-835a-78c4540ba5b2 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:59.060950 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046fb59-10fa-4714-88eb-e896edd30f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:59.062137 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046fb59-10fa-4714-88eb-e896edd30f14 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:59.062602 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046fb59-10fa-4714-88eb-e896edd30f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:59.063072 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046fb59-10fa-4714-88eb-e896edd30f14 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:59.064395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4046fb59-10fa-4714-88eb-e896edd30f14 None admin] RBAC: Adding query filter params (interface=public, service_id=44004a3de11c4b88b3f37663489180a6, region_id=RegionOne) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:59.064983 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4046fb59-10fa-4714-88eb-e896edd30f14 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:59.067503 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4046fb59-10fa-4714-88eb-e896edd30f14 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:59.078813 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 451/910] 10.222.0.44 () {56 vars in 1251 bytes} [Sun Sep 11 14:59:58 2022] GET /identity/v3/endpoints?service_id=44004a3de11c4b88b3f37663489180a6&interface=public®ion_id=RegionOne => generated 194 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:59.086299 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046fb59-10fa-4714-88eb-e896edd30f14 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 14:59:59.110878 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f256d-c36c-4621-8bee-7aa7f7ec15d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:59.112784 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f256d-c36c-4621-8bee-7aa7f7ec15d5 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:59.113557 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f256d-c36c-4621-8bee-7aa7f7ec15d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:59.114350 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f256d-c36c-4621-8bee-7aa7f7ec15d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:59.115792 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f256d-c36c-4621-8bee-7aa7f7ec15d5 None admin] RBAC: Adding query filter params (role.id=92fb89d658dc4d4788cef145280a11f7, scope.project.id=70ac515938a24c45ae6302dbac46a1ab, user.id=3fada78ceaeb4f3bad77c516f6dfcfd4) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 14:59:59.116821 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f256d-c36c-4621-8bee-7aa7f7ec15d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:59.120585 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f256d-c36c-4621-8bee-7aa7f7ec15d5 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:59.130360 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 460/911] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 14:59:58 2022] GET /identity/v3/role_assignments?user.id=3fada78ceaeb4f3bad77c516f6dfcfd4&scope.project.id=70ac515938a24c45ae6302dbac46a1ab&role.id=92fb89d658dc4d4788cef145280a11f7 => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 14:59:59.229939 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 14:59:59.283842 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046fb59-10fa-4714-88eb-e896edd30f14 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 14:59:59.327765 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c1aac-3b33-4c20-aa61-4152c74f44b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 14:59:59.328296 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c1aac-3b33-4c20-aa61-4152c74f44b1 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 14:59:59.328575 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c1aac-3b33-4c20-aa61-4152c74f44b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 14:59:59.328922 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c1aac-3b33-4c20-aa61-4152c74f44b1 None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 14:59:59.329611 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c1aac-3b33-4c20-aa61-4152c74f44b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 14:59:59.331157 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c1aac-3b33-4c20-aa61-4152c74f44b1 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 14:59:59.339253 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 452/912] 10.222.0.44 () {56 vars in 1087 bytes} [Sun Sep 11 14:59:59 2022] GET /identity/v3/services => generated 1497 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:00:00.202721 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e748c8c-6c84-4502-ba7b-ca097528d7d3 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:00.203565 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e748c8c-6c84-4502-ba7b-ca097528d7d3 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:00.203565 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e748c8c-6c84-4502-ba7b-ca097528d7d3 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:00.204950 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 461/913] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:00.211981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18779150-cc78-4cbf-9d81-4a92efd92004 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:00.212509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18779150-cc78-4cbf-9d81-4a92efd92004 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:00.212884 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18779150-cc78-4cbf-9d81-4a92efd92004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:00.246384 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-18779150-cc78-4cbf-9d81-4a92efd92004 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:00.248872 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-18779150-cc78-4cbf-9d81-4a92efd92004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:00.375859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 453/914] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:00:00.874994 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e748c8c-6c84-4502-ba7b-ca097528d7d3 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:01.072751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e748c8c-6c84-4502-ba7b-ca097528d7d3 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:01.117421 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:01.129377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea955ef-1226-4266-a76e-373bc9aded7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:01.130451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea955ef-1226-4266-a76e-373bc9aded7e None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:01.131238 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea955ef-1226-4266-a76e-373bc9aded7e None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:01.131797 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea955ef-1226-4266-a76e-373bc9aded7e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:01.132611 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea955ef-1226-4266-a76e-373bc9aded7e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:01.134054 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea955ef-1226-4266-a76e-373bc9aded7e None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:01.145408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-eea955ef-1226-4266-a76e-373bc9aded7e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 15:00:01.145408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:01.145408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 15:00:01.145408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 15:00:01.145408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 15:00:01.145408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 15:00:01.145408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 15:00:01.145408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:00:01.145408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 15:00:01.145408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:01.145408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 15:00:01.145408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:01.145408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:01.145408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:01.145408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:01.145408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:01.145408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:01.147529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:01.147529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:01.147529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:01.147529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:01.147529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:01.147529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:01.147529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 15:00:01.147529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 15:00:01.147529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 15:00:01.147529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 15:00:01.147529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:01.147529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:01.147529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:01.147529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:01.147529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:01.147529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:01.148893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:01.148893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:01.148893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:01.148893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:01.148893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:01.148893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:01.148893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:01.148893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:01.148893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:01.148893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:01.148893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 15:00:01.148893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:00:01.148893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 15:00:01.148893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:01.148893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 462/915] 10.222.0.44 () {56 vars in 1126 bytes} [Sun Sep 11 15:00:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:01.161793 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18779150-cc78-4cbf-9d81-4a92efd92004 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:01.293893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:01.330965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18779150-cc78-4cbf-9d81-4a92efd92004 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:01.369366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce74749-8e77-4290-81fb-9be556a0f73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:01.370045 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce74749-8e77-4290-81fb-9be556a0f73c None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:01.370322 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce74749-8e77-4290-81fb-9be556a0f73c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:01.370593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce74749-8e77-4290-81fb-9be556a0f73c None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:01.371241 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce74749-8e77-4290-81fb-9be556a0f73c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:01.371539 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce74749-8e77-4290-81fb-9be556a0f73c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:01.372966 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce74749-8e77-4290-81fb-9be556a0f73c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:01.379000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 454/916] 10.222.0.44 () {56 vars in 1122 bytes} [Sun Sep 11 15:00:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:01.388163 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea955ef-1226-4266-a76e-373bc9aded7e None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:01.541585 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea955ef-1226-4266-a76e-373bc9aded7e None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:01.578746 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efe673-6f59-4e6f-bbe5-a4ee77bf7a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:01.579378 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efe673-6f59-4e6f-bbe5-a4ee77bf7a4a None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:01.579681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efe673-6f59-4e6f-bbe5-a4ee77bf7a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:01.579921 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efe673-6f59-4e6f-bbe5-a4ee77bf7a4a None admin] PATH_INFO: `/v3/services` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:01.580554 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18efe673-6f59-4e6f-bbe5-a4ee77bf7a4a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:01.580858 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18efe673-6f59-4e6f-bbe5-a4ee77bf7a4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:01.582313 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18efe673-6f59-4e6f-bbe5-a4ee77bf7a4a None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:01.588404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 463/917] 10.222.0.44 () {56 vars in 1122 bytes} [Sun Sep 11 15:00:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:01.598064 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce74749-8e77-4290-81fb-9be556a0f73c None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:01.718391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce74749-8e77-4290-81fb-9be556a0f73c None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:01.752802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:01.762605 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f12666-f8c3-4536-945c-3d0e7cda5512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:01.763253 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f12666-f8c3-4536-945c-3d0e7cda5512 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:01.763535 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f12666-f8c3-4536-945c-3d0e7cda5512 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:01.763781 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f12666-f8c3-4536-945c-3d0e7cda5512 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:01.764513 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f12666-f8c3-4536-945c-3d0e7cda5512 None admin] RBAC: Adding query filter params (interface=public, service_id=a123fbac018f43e7a264df85406f2166, region_id=RegionOne) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:01.764821 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f12666-f8c3-4536-945c-3d0e7cda5512 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:01.766314 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f12666-f8c3-4536-945c-3d0e7cda5512 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:01.772864 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 455/918] 10.222.0.44 () {56 vars in 1251 bytes} [Sun Sep 11 15:00:01 2022] GET /identity/v3/endpoints?service_id=a123fbac018f43e7a264df85406f2166&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:01.783602 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efe673-6f59-4e6f-bbe5-a4ee77bf7a4a None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:01.879190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:01.937822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efe673-6f59-4e6f-bbe5-a4ee77bf7a4a None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:01.983388 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ada71-82ac-421c-a623-8b8e651d8507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:01.984025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ada71-82ac-421c-a623-8b8e651d8507 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:01.984369 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ada71-82ac-421c-a623-8b8e651d8507 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:01.984706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ada71-82ac-421c-a623-8b8e651d8507 None admin] PATH_INFO: `/v3/services` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:01.985439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ada71-82ac-421c-a623-8b8e651d8507 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:01.986919 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ada71-82ac-421c-a623-8b8e651d8507 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:01.994061 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 464/919] 10.222.0.44 () {56 vars in 1087 bytes} [Sun Sep 11 15:00:01 2022] GET /identity/v3/services => generated 1497 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:00:02.048342 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a4d36-69b5-41e1-a20b-230ff0269e6a None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:02.048726 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a4d36-69b5-41e1-a20b-230ff0269e6a None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:02.049074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a4d36-69b5-41e1-a20b-230ff0269e6a None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:02.050656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 456/920] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:02.056918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20800e96-58c0-44db-be0f-f8833750c826 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:02.057389 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20800e96-58c0-44db-be0f-f8833750c826 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:02.057737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20800e96-58c0-44db-be0f-f8833750c826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:02.078507 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-20800e96-58c0-44db-be0f-f8833750c826 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:02.079724 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c24989-ccf5-415a-a603-5e6cea8ee96d None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:02.080002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-20800e96-58c0-44db-be0f-f8833750c826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:02.080355 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c24989-ccf5-415a-a603-5e6cea8ee96d None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:02.082564 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c24989-ccf5-415a-a603-5e6cea8ee96d None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:02.084501 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 457/921] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:02.091107 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe800d-b037-4740-b68f-eca250470523 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:02.091787 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe800d-b037-4740-b68f-eca250470523 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:02.092443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe800d-b037-4740-b68f-eca250470523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:02.125740 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-3afe800d-b037-4740-b68f-eca250470523 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:02.128076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-3afe800d-b037-4740-b68f-eca250470523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:02.171113 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 465/922] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:00:02.259409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 458/923] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:00:02.281257 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f1f28-11cc-49cb-931a-63c5bb0d0fe1 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:02.284185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f1f28-11cc-49cb-931a-63c5bb0d0fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:02.284438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f1f28-11cc-49cb-931a-63c5bb0d0fe1 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:02.289412 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 466/924] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:02.296925 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721d264-b3b4-4d14-8528-ba890e28a8f3 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:02.297684 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721d264-b3b4-4d14-8528-ba890e28a8f3 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:02.298195 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721d264-b3b4-4d14-8528-ba890e28a8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:02.330606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-0721d264-b3b4-4d14-8528-ba890e28a8f3 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:02.332593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-0721d264-b3b4-4d14-8528-ba890e28a8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:02.447702 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f1f28-11cc-49cb-931a-63c5bb0d0fe1 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:02.473267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 459/925] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:00:02.521062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:02.603246 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f1f28-11cc-49cb-931a-63c5bb0d0fe1 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:02.668895 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e0707-854e-46c8-a221-bb833f4e1bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:02.670029 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e0707-854e-46c8-a221-bb833f4e1bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:02.670504 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e0707-854e-46c8-a221-bb833f4e1bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:02.670993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e0707-854e-46c8-a221-bb833f4e1bb7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:02.691675 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963e0707-854e-46c8-a221-bb833f4e1bb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 15:00:02.701221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-963e0707-854e-46c8-a221-bb833f4e1bb7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 15:00:02.701221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:02.701221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:02.701221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:02.701221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:02.701221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:02.701221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:02.701221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:02.701221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:02.701221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:02.701221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:02.701221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:02.701221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 15:00:02.701221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 15:00:02.701221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 15:00:02.701221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 15:00:02.704215 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 15:00:02.704215 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 15:00:02.704215 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 15:00:02.704215 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 15:00:02.704215 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:02.704215 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:02.704215 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:02.704215 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:02.704215 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:02.704215 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:02.704215 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:02.704215 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:02.704215 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:02.704215 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:02.704215 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:02.704215 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:02.705648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:02.705648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:02.705648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:02.705648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:02.705648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 15:00:02.705648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 15:00:02.705648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 15:00:02.705648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 15:00:02.705648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 15:00:02.705648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 15:00:02.705648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 15:00:02.705648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:02.705648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 467/926] 10.222.0.44 () {56 vars in 1120 bytes} [Sun Sep 11 15:00:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:02.711150 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721d264-b3b4-4d14-8528-ba890e28a8f3 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:02.715999 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e0707-854e-46c8-a221-bb833f4e1bb7 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:02.824373 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:02.901249 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e0707-854e-46c8-a221-bb833f4e1bb7 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:02.914470 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:02.921191 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721d264-b3b4-4d14-8528-ba890e28a8f3 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:02.961280 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520287b-439b-4413-bc10-5079909880f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:02.962800 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520287b-439b-4413-bc10-5079909880f7 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:02.963278 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520287b-439b-4413-bc10-5079909880f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:02.963823 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520287b-439b-4413-bc10-5079909880f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:02.965025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520287b-439b-4413-bc10-5079909880f7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:02.965636 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520287b-439b-4413-bc10-5079909880f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:02.968303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520287b-439b-4413-bc10-5079909880f7 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:02.969193 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c842486-0383-4fe3-bf06-6e262acd46d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:02.970270 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c842486-0383-4fe3-bf06-6e262acd46d6 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:02.971214 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c842486-0383-4fe3-bf06-6e262acd46d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:02.971527 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c842486-0383-4fe3-bf06-6e262acd46d6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:02.972585 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c842486-0383-4fe3-bf06-6e262acd46d6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:02.975242 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c842486-0383-4fe3-bf06-6e262acd46d6 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:02.987810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-0c842486-0383-4fe3-bf06-6e262acd46d6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 15:00:02.987810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:02.987810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 15:00:02.987810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 15:00:02.987810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 15:00:02.987810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 15:00:02.987810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 15:00:02.987810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:00:02.987810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 15:00:02.987810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:02.987810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 15:00:02.987810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:02.987810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:02.987810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:02.987810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:02.987810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:02.987810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:02.993978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:02.993978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:02.993978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:02.993978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:02.993978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:02.993978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:02.993978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 15:00:02.993978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 15:00:02.993978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 15:00:02.993978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 15:00:02.993978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:02.993978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:02.993978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:02.993978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:02.993978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:02.993978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:02.994954 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:02.994954 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:02.994954 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:02.994954 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:02.994954 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:02.994954 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:02.994954 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:02.994954 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:02.994954 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:02.994954 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:02.994954 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 15:00:02.994954 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:00:02.994954 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 15:00:02.994954 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:02.994954 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 460/927] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 15:00:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:02.995769 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 468/928] 10.222.0.44 () {56 vars in 1115 bytes} [Sun Sep 11 15:00:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:02.995845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c842486-0383-4fe3-bf06-6e262acd46d6 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:02.995915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520287b-439b-4413-bc10-5079909880f7 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:03.139099 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c842486-0383-4fe3-bf06-6e262acd46d6 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:03.156956 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:03.164069 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520287b-439b-4413-bc10-5079909880f7 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:03.177462 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f08a45-cb91-49d2-bd15-2ee2a3012f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:03.178136 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f08a45-cb91-49d2-bd15-2ee2a3012f52 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:03.178384 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f08a45-cb91-49d2-bd15-2ee2a3012f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:03.178662 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f08a45-cb91-49d2-bd15-2ee2a3012f52 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:03.179248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f08a45-cb91-49d2-bd15-2ee2a3012f52 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:03.180621 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f08a45-cb91-49d2-bd15-2ee2a3012f52 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:03.192383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-01f08a45-cb91-49d2-bd15-2ee2a3012f52 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 15:00:03.192383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:03.192383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 15:00:03.192383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 15:00:03.192383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 15:00:03.192383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 15:00:03.192383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 15:00:03.192383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:00:03.192383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 15:00:03.192383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:03.192383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 15:00:03.192383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:03.192383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:03.192383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:03.192383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:03.192383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:03.194981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:03.194981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:03.194981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:03.194981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:03.194981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:03.194981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:03.194981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:03.194981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 15:00:03.194981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 15:00:03.194981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 15:00:03.194981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 15:00:03.194981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:03.194981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:03.194981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:03.194981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:03.194981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:03.194981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:03.196748 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:03.196748 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:03.196748 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:03.196748 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:03.196748 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:03.196748 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:03.196748 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:03.196748 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:03.196748 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:03.196748 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:03.196748 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 15:00:03.196748 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:00:03.196748 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 15:00:03.196748 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:03.196748 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 461/929] 10.222.0.44 () {56 vars in 1132 bytes} [Sun Sep 11 15:00:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:03.201301 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f08a45-cb91-49d2-bd15-2ee2a3012f52 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:03.219013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab5ebb-a118-4a70-8052-1d369f5db05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:03.221106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab5ebb-a118-4a70-8052-1d369f5db05a None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:03.221693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab5ebb-a118-4a70-8052-1d369f5db05a None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:03.222168 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab5ebb-a118-4a70-8052-1d369f5db05a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:03.241881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab5ebb-a118-4a70-8052-1d369f5db05a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:03.244577 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab5ebb-a118-4a70-8052-1d369f5db05a None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:03.274123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-18ab5ebb-a118-4a70-8052-1d369f5db05a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:00:03.274123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:03.274123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:03.274123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:03.274123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:03.274123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:03.274123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:03.274123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:03.274123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:03.274123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:03.274123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:03.274123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:03.274123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 15:00:03.274123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 15:00:03.274123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 15:00:03.274123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 15:00:03.275170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:03.275170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:03.275170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:03.275170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:03.275170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:03.275170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:03.275170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:03.275170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:03.275170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:03.275170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:03.275170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:03.275170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:03.275170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:03.275170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:03.275170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:03.275170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:03.275170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 15:00:03.276153 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 15:00:03.276153 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:00:03.276153 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:03.278879 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 469/930] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:00:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 15:00:03.280475 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab5ebb-a118-4a70-8052-1d369f5db05a None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:03.321014 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:03.344664 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f08a45-cb91-49d2-bd15-2ee2a3012f52 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:03.386460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524dcf7-a09b-4308-af74-8eb626a793bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:03.387392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524dcf7-a09b-4308-af74-8eb626a793bc None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:03.387654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524dcf7-a09b-4308-af74-8eb626a793bc None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:03.387926 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524dcf7-a09b-4308-af74-8eb626a793bc None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:03.388587 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c524dcf7-a09b-4308-af74-8eb626a793bc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:03.388885 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c524dcf7-a09b-4308-af74-8eb626a793bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:03.390502 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c524dcf7-a09b-4308-af74-8eb626a793bc None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:03.398680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 462/931] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 15:00:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:03.399295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524dcf7-a09b-4308-af74-8eb626a793bc None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:03.430279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab5ebb-a118-4a70-8052-1d369f5db05a None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:03.468924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21799ef5-e634-41c5-9d32-a755f1af8ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:03.469719 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21799ef5-e634-41c5-9d32-a755f1af8ecf None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:03.469992 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21799ef5-e634-41c5-9d32-a755f1af8ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:03.470262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21799ef5-e634-41c5-9d32-a755f1af8ecf None admin] PATH_INFO: `/v3/services` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:03.471268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21799ef5-e634-41c5-9d32-a755f1af8ecf None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:03.471569 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21799ef5-e634-41c5-9d32-a755f1af8ecf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:03.472982 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21799ef5-e634-41c5-9d32-a755f1af8ecf None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:03.484743 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 470/932] 10.222.0.44 () {56 vars in 1126 bytes} [Sun Sep 11 15:00:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:03.484937 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:03.485018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21799ef5-e634-41c5-9d32-a755f1af8ecf None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:03.574225 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524dcf7-a09b-4308-af74-8eb626a793bc None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:03.618715 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4424d6-f8c8-4404-84b7-8839ef4422cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:03.619943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4424d6-f8c8-4404-84b7-8839ef4422cb None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:03.620512 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4424d6-f8c8-4404-84b7-8839ef4422cb None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:03.620851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4424d6-f8c8-4404-84b7-8839ef4422cb None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:03.621775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4424d6-f8c8-4404-84b7-8839ef4422cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:03.622379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4424d6-f8c8-4404-84b7-8839ef4422cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:03.624226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4424d6-f8c8-4404-84b7-8839ef4422cb None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:03.628121 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:03.639799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 463/933] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 15:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:03.641054 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4424d6-f8c8-4404-84b7-8839ef4422cb None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:03.650739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21799ef5-e634-41c5-9d32-a755f1af8ecf None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:03.695168 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b3108-f8b6-4866-95a0-799b8c879e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:03.696017 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b3108-f8b6-4866-95a0-799b8c879e6f None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:03.696410 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b3108-f8b6-4866-95a0-799b8c879e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:03.696852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b3108-f8b6-4866-95a0-799b8c879e6f None admin] PATH_INFO: `/v3/services` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:03.697981 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2b3108-f8b6-4866-95a0-799b8c879e6f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:03.698459 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2b3108-f8b6-4866-95a0-799b8c879e6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:03.700634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2b3108-f8b6-4866-95a0-799b8c879e6f None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:03.713881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 471/934] 10.222.0.44 () {56 vars in 1114 bytes} [Sun Sep 11 15:00:03 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:03.715160 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b3108-f8b6-4866-95a0-799b8c879e6f None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:03.773566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4424d6-f8c8-4404-84b7-8839ef4422cb None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:03.776778 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:03.819154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ab44c-5aec-4cec-84be-e6014e649810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:03.819770 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ab44c-5aec-4cec-84be-e6014e649810 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:03.820043 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ab44c-5aec-4cec-84be-e6014e649810 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:03.820299 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ab44c-5aec-4cec-84be-e6014e649810 None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:03.820926 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855ab44c-5aec-4cec-84be-e6014e649810 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:03.821255 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855ab44c-5aec-4cec-84be-e6014e649810 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:03.822637 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855ab44c-5aec-4cec-84be-e6014e649810 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:03.828348 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 464/935] 10.222.0.44 () {56 vars in 1126 bytes} [Sun Sep 11 15:00:03 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:03.829348 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ab44c-5aec-4cec-84be-e6014e649810 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:03.851362 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b3108-f8b6-4866-95a0-799b8c879e6f None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:03.897763 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f3b0a5-8cd1-47bf-8f48-b0c417d787fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:03.898315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3b0a5-8cd1-47bf-8f48-b0c417d787fd None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:03.898648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3b0a5-8cd1-47bf-8f48-b0c417d787fd None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:03.898889 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3b0a5-8cd1-47bf-8f48-b0c417d787fd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:03.907502 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f3b0a5-8cd1-47bf-8f48-b0c417d787fd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:03.908948 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f3b0a5-8cd1-47bf-8f48-b0c417d787fd None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:03.921934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-83f3b0a5-8cd1-47bf-8f48-b0c417d787fd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 15:00:03.921934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:03.921934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:03.921934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:03.921934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:03.921934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:03.921934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:03.921934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:03.921934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:03.921934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:03.921934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:03.921934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:03.921934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 15:00:03.921934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 15:00:03.921934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 15:00:03.921934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 15:00:03.923354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:03.923354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:03.923354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 15:00:03.923354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 15:00:03.923354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 15:00:03.923354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 15:00:03.923354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:03.923354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:03.923354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:03.923354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:03.923354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:03.923354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:03.923354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:03.923354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:03.923354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:03.923354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:03.923354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:03.923354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:03.925124 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:03.925124 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:03.925124 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 15:00:03.925124 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 15:00:03.925124 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 15:00:03.925124 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 15:00:03.925124 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 15:00:03.925124 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 15:00:03.925124 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 15:00:03.925124 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:03.925124 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 472/936] 10.222.0.44 () {56 vars in 1117 bytes} [Sun Sep 11 15:00:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:03.925124 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f3b0a5-8cd1-47bf-8f48-b0c417d787fd None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:03.941059 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:03.968731 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ab44c-5aec-4cec-84be-e6014e649810 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:04.010215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21804d9a-5508-47ea-83b1-ef3c577b7e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:04.010900 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21804d9a-5508-47ea-83b1-ef3c577b7e65 None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:04.011168 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21804d9a-5508-47ea-83b1-ef3c577b7e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:04.011431 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21804d9a-5508-47ea-83b1-ef3c577b7e65 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:04.012110 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21804d9a-5508-47ea-83b1-ef3c577b7e65 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:04.013544 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21804d9a-5508-47ea-83b1-ef3c577b7e65 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:04.045641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 465/937] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:00:03 2022] POST /identity/v3/endpoints => generated 346 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:00:04.047126 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21804d9a-5508-47ea-83b1-ef3c577b7e65 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:04.066109 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f3b0a5-8cd1-47bf-8f48-b0c417d787fd None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:04.085932 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:04.163009 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89e0e2-66ce-443c-8042-81bff0a21c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:04.163941 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89e0e2-66ce-443c-8042-81bff0a21c1b None admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:04.164798 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89e0e2-66ce-443c-8042-81bff0a21c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:04.165284 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89e0e2-66ce-443c-8042-81bff0a21c1b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:04.166594 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f89e0e2-66ce-443c-8042-81bff0a21c1b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:04.168607 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f89e0e2-66ce-443c-8042-81bff0a21c1b None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:04.195067 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21804d9a-5508-47ea-83b1-ef3c577b7e65 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:04.205562 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 473/938] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:00:03 2022] POST /identity/v3/endpoints => generated 347 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:00:04.266373 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2b7b6-98f5-4624-941b-eab4bf3d0245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:04.267616 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2b7b6-98f5-4624-941b-eab4bf3d0245 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:04.268466 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2b7b6-98f5-4624-941b-eab4bf3d0245 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:04.268956 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2b7b6-98f5-4624-941b-eab4bf3d0245 None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:04.270511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2b7b6-98f5-4624-941b-eab4bf3d0245 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:04.275537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2b7b6-98f5-4624-941b-eab4bf3d0245 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:04.278803 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2b7b6-98f5-4624-941b-eab4bf3d0245 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:04.316608 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 466/939] 10.222.0.44 () {56 vars in 1149 bytes} [Sun Sep 11 15:00:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:04.323276 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89e0e2-66ce-443c-8042-81bff0a21c1b None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:04.404582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:04.457007 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d5fa7-46d8-44d3-b46e-b8f0f8806db8 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:04.457547 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89e0e2-66ce-443c-8042-81bff0a21c1b None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:04.457630 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d5fa7-46d8-44d3-b46e-b8f0f8806db8 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:04.458077 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d5fa7-46d8-44d3-b46e-b8f0f8806db8 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:04.460586 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 467/940] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:04.468484 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3a370-6e37-4200-98b4-91c68ff27e6f None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:04.468861 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3a370-6e37-4200-98b4-91c68ff27e6f None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:04.469175 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3a370-6e37-4200-98b4-91c68ff27e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:04.494917 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-c4f3a370-6e37-4200-98b4-91c68ff27e6f None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:04.496779 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c4f3a370-6e37-4200-98b4-91c68ff27e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:04.510417 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c07c93-627a-42ee-ae78-4fea5dcf88ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:04.511514 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c07c93-627a-42ee-ae78-4fea5dcf88ad None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:04.511999 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c07c93-627a-42ee-ae78-4fea5dcf88ad None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:04.512486 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c07c93-627a-42ee-ae78-4fea5dcf88ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:04.531076 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c07c93-627a-42ee-ae78-4fea5dcf88ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:04.532968 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c07c93-627a-42ee-ae78-4fea5dcf88ad None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:04.549870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-b6c07c93-627a-42ee-ae78-4fea5dcf88ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:00:04.549870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:04.549870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:04.549870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:04.549870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:04.549870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:04.549870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:04.549870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:04.549870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:04.549870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:04.549870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:04.549870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:04.549870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 15:00:04.549870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 15:00:04.549870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 15:00:04.549870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 15:00:04.550881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:04.550881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:04.550881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:04.550881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:04.550881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:04.550881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:04.550881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:04.550881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:04.550881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:04.550881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:04.550881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:04.550881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:04.550881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:04.550881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:04.550881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:04.550881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:04.550881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 15:00:04.551752 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 15:00:04.551752 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:00:04.551752 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:04.554103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 474/941] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:00:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:04.561789 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c07c93-627a-42ee-ae78-4fea5dcf88ad None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:04.636502 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 468/942] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 15:00:04.692040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c07c93-627a-42ee-ae78-4fea5dcf88ad None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:04.708245 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:04.732545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dac512-4391-4af8-a4f4-cd685da9339c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:04.733201 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dac512-4391-4af8-a4f4-cd685da9339c None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:04.733485 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dac512-4391-4af8-a4f4-cd685da9339c None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:04.733757 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dac512-4391-4af8-a4f4-cd685da9339c None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:04.734477 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dac512-4391-4af8-a4f4-cd685da9339c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:04.734784 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dac512-4391-4af8-a4f4-cd685da9339c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:04.736242 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dac512-4391-4af8-a4f4-cd685da9339c None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:04.750861 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 475/943] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 15:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:04.756517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3a370-6e37-4200-98b4-91c68ff27e6f None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:04.768790 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:04.896955 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3a370-6e37-4200-98b4-91c68ff27e6f None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:04.910875 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dac512-4391-4af8-a4f4-cd685da9339c None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:04.943777 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099fcd80-e0b4-4a81-a982-f76c40d50a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:04.944626 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fcd80-e0b4-4a81-a982-f76c40d50a57 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:04.945016 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fcd80-e0b4-4a81-a982-f76c40d50a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:04.945338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fcd80-e0b4-4a81-a982-f76c40d50a57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:04.954240 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099fcd80-e0b4-4a81-a982-f76c40d50a57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:04.955786 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099fcd80-e0b4-4a81-a982-f76c40d50a57 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:04.969685 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-099fcd80-e0b4-4a81-a982-f76c40d50a57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 15:00:04.969685 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:04.969685 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:04.969685 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:04.969685 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:04.969685 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:04.969685 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:04.969685 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:04.969685 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:04.969685 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:04.969685 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:04.969685 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:04.969685 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 15:00:04.969685 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 15:00:04.969685 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 15:00:04.971692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 15:00:04.971692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:04.971692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:04.971692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:04.971692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:04.971692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:04.971692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:04.971692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:04.971692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:04.971692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:04.971692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:04.971692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:04.971692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:04.971692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:04.971692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:04.971692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:04.971692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:04.973378 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 15:00:04.973378 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 15:00:04.973378 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 15:00:04.973378 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 15:00:04.973378 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 15:00:04.973378 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 15:00:04.973378 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 15:00:04.973378 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:04.973378 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 469/944] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 15:00:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:04.983907 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099fcd80-e0b4-4a81-a982-f76c40d50a57 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:05.034378 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dac512-4391-4af8-a4f4-cd685da9339c None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:05.074219 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65778cb7-066d-4264-9f19-485d527d7260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:05.074880 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65778cb7-066d-4264-9f19-485d527d7260 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:05.075156 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65778cb7-066d-4264-9f19-485d527d7260 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:05.075423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65778cb7-066d-4264-9f19-485d527d7260 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:05.076028 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65778cb7-066d-4264-9f19-485d527d7260 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:05.077587 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65778cb7-066d-4264-9f19-485d527d7260 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:05.087497 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-65778cb7-066d-4264-9f19-485d527d7260 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 15:00:05.087497 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:05.087497 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 15:00:05.087497 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 15:00:05.087497 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 15:00:05.087497 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 15:00:05.087497 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 15:00:05.087497 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:00:05.087497 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 15:00:05.087497 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:05.087497 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 15:00:05.087497 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:05.087497 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:05.087497 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:05.087497 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:05.087497 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:05.087497 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:05.089512 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:05.089512 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:05.089512 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:05.089512 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:05.089512 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:05.089512 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:05.089512 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 15:00:05.089512 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 15:00:05.089512 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 15:00:05.089512 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 15:00:05.089512 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:05.089512 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:05.089512 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:05.089512 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:05.089512 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:05.089512 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:05.091111 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:05.091111 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:05.091111 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:05.091111 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:05.091111 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:05.091111 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:05.091111 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:05.091111 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:05.091111 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:05.091111 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:05.091111 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 15:00:05.091111 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:00:05.091111 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 15:00:05.091111 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:05.094110 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 476/945] 10.222.0.44 () {56 vars in 1126 bytes} [Sun Sep 11 15:00:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 15:00:05.095172 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65778cb7-066d-4264-9f19-485d527d7260 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:05.122229 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:05.167486 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:05.232999 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099fcd80-e0b4-4a81-a982-f76c40d50a57 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:05.249207 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65778cb7-066d-4264-9f19-485d527d7260 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:05.304710 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22665d9-a167-425a-a5ee-633d90e05ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:05.305835 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22665d9-a167-425a-a5ee-633d90e05ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:05.306334 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22665d9-a167-425a-a5ee-633d90e05ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:05.306830 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22665d9-a167-425a-a5ee-633d90e05ef8 None admin] PATH_INFO: `/v3/services` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:05.307708 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22665d9-a167-425a-a5ee-633d90e05ef8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:05.308106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22665d9-a167-425a-a5ee-633d90e05ef8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:05.309696 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22665d9-a167-425a-a5ee-633d90e05ef8 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:05.315925 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 477/946] 10.222.0.44 () {56 vars in 1122 bytes} [Sun Sep 11 15:00:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:05.321956 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22665d9-a167-425a-a5ee-633d90e05ef8 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:05.323949 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d834bdc-1ebe-4851-a2cc-9e15ca2b6e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:05.325484 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d834bdc-1ebe-4851-a2cc-9e15ca2b6e03 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:05.325811 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d834bdc-1ebe-4851-a2cc-9e15ca2b6e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:05.326056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d834bdc-1ebe-4851-a2cc-9e15ca2b6e03 None admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:05.326747 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d834bdc-1ebe-4851-a2cc-9e15ca2b6e03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:05.327046 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d834bdc-1ebe-4851-a2cc-9e15ca2b6e03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:05.328543 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d834bdc-1ebe-4851-a2cc-9e15ca2b6e03 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:05.352238 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 470/947] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 15:00:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:05.358753 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d834bdc-1ebe-4851-a2cc-9e15ca2b6e03 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:05.479883 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22665d9-a167-425a-a5ee-633d90e05ef8 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:05.484232 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:05.545022 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71491dd7-be47-43da-b6e2-ee6192b11a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:05.546084 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71491dd7-be47-43da-b6e2-ee6192b11a73 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:05.546084 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71491dd7-be47-43da-b6e2-ee6192b11a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:05.546333 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71491dd7-be47-43da-b6e2-ee6192b11a73 None admin] PATH_INFO: `/v3/services` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:05.546906 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71491dd7-be47-43da-b6e2-ee6192b11a73 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:05.547205 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71491dd7-be47-43da-b6e2-ee6192b11a73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:05.548611 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71491dd7-be47-43da-b6e2-ee6192b11a73 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:05.559018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 478/948] 10.222.0.44 () {56 vars in 1122 bytes} [Sun Sep 11 15:00:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:05.559962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d834bdc-1ebe-4851-a2cc-9e15ca2b6e03 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:05.560683 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71491dd7-be47-43da-b6e2-ee6192b11a73 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:05.615273 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:05.632008 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638745d-113f-4478-a5fe-5846077e39b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:05.633800 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638745d-113f-4478-a5fe-5846077e39b8 None admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:05.634696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638745d-113f-4478-a5fe-5846077e39b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:05.635545 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638745d-113f-4478-a5fe-5846077e39b8 None admin] PATH_INFO: `/v3/projects/70ac515938a24c45ae6302dbac46a1ab/users/3fada78ceaeb4f3bad77c516f6dfcfd4/roles/92fb89d658dc4d4788cef145280a11f7` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:05.640060 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638745d-113f-4478-a5fe-5846077e39b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=70ac515938a24c45ae6302dbac46a1ab, user_id=3fada78ceaeb4f3bad77c516f6dfcfd4, role_id=92fb89d658dc4d4788cef145280a11f7)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:05.643506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638745d-113f-4478-a5fe-5846077e39b8 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:05.689488 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 471/949] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 15:00:05 2022] PUT /identity/v3/projects/70ac515938a24c45ae6302dbac46a1ab/users/3fada78ceaeb4f3bad77c516f6dfcfd4/roles/92fb89d658dc4d4788cef145280a11f7 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:00:05.692139 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71491dd7-be47-43da-b6e2-ee6192b11a73 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:05.737048 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1339e0b-ab10-42b4-9934-47daf6652320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:05.738058 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1339e0b-ab10-42b4-9934-47daf6652320 None admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:05.738493 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1339e0b-ab10-42b4-9934-47daf6652320 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:05.738912 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1339e0b-ab10-42b4-9934-47daf6652320 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:05.739762 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1339e0b-ab10-42b4-9934-47daf6652320 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:05.741507 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1339e0b-ab10-42b4-9934-47daf6652320 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:05.769638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 479/950] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:00:05 2022] POST /identity/v3/endpoints => generated 349 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:00:06.694041 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5b16d-fa2e-44f8-b474-0fc9ec29c083 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:06.694578 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5b16d-fa2e-44f8-b474-0fc9ec29c083 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:06.694578 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5b16d-fa2e-44f8-b474-0fc9ec29c083 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:06.696255 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 472/951] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:06.702633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b687750-1792-491e-a5b6-db22e1cdad88 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:06.703219 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b687750-1792-491e-a5b6-db22e1cdad88 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:06.703219 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b687750-1792-491e-a5b6-db22e1cdad88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:06.724135 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-8b687750-1792-491e-a5b6-db22e1cdad88 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:06.725405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-8b687750-1792-491e-a5b6-db22e1cdad88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:06.744428 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:06.817298 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 480/952] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:00:07.078566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa5b16d-fa2e-44f8-b474-0fc9ec29c083 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:07.234449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa5b16d-fa2e-44f8-b474-0fc9ec29c083 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:07.301709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5d865-55c3-4c43-94b6-f5912d654051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:07.302852 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5d865-55c3-4c43-94b6-f5912d654051 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:07.303340 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5d865-55c3-4c43-94b6-f5912d654051 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:07.303810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5d865-55c3-4c43-94b6-f5912d654051 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:07.304946 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5d865-55c3-4c43-94b6-f5912d654051 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:07.307660 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5d865-55c3-4c43-94b6-f5912d654051 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:07.325641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-d3f5d865-55c3-4c43-94b6-f5912d654051 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 15:00:07.325641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:07.325641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 15:00:07.325641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 15:00:07.325641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 15:00:07.325641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 15:00:07.325641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 15:00:07.325641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:00:07.325641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 15:00:07.325641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:07.325641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 15:00:07.325641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:07.325641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:07.325641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:07.325641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:07.325641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:07.325641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:07.326776 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:07.326776 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:07.326776 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:07.326776 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:07.326776 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:07.326776 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:07.326776 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 15:00:07.326776 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 15:00:07.326776 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 15:00:07.326776 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 15:00:07.326776 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:07.326776 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:07.326776 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:07.326776 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:07.326776 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:07.326776 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:07.327701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:07.327701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:07.327701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:07.327701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:07.327701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:07.327701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:07.327701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:07.327701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:07.327701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:07.327701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:07.327701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 15:00:07.327701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:00:07.327701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 15:00:07.327701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:07.328491 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 473/953] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 15:00:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:07.334294 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b687750-1792-491e-a5b6-db22e1cdad88 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:07.450140 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b687750-1792-491e-a5b6-db22e1cdad88 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:07.490198 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0c4adb-b2a3-468e-9223-48d88e985e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:07.491179 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c4adb-b2a3-468e-9223-48d88e985e01 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:07.491645 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c4adb-b2a3-468e-9223-48d88e985e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:07.492090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c4adb-b2a3-468e-9223-48d88e985e01 None admin] PATH_INFO: `/v3/services` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:07.492908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0c4adb-b2a3-468e-9223-48d88e985e01 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:07.493428 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0c4adb-b2a3-468e-9223-48d88e985e01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:07.495090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0c4adb-b2a3-468e-9223-48d88e985e01 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:07.501003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 481/954] 10.222.0.44 () {56 vars in 1112 bytes} [Sun Sep 11 15:00:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:07.509754 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5d865-55c3-4c43-94b6-f5912d654051 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:07.593565 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:07.615925 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abfcbd-2056-491f-8c43-26dbb0246e72 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:07.616180 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abfcbd-2056-491f-8c43-26dbb0246e72 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:07.616532 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abfcbd-2056-491f-8c43-26dbb0246e72 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:07.618128 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 482/955] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 15:00:07.624857 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d8e42-a0cf-4b1b-9c75-3ab8524b4d39 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:07.625363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d8e42-a0cf-4b1b-9c75-3ab8524b4d39 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:07.625817 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d8e42-a0cf-4b1b-9c75-3ab8524b4d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:07.653506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5d865-55c3-4c43-94b6-f5912d654051 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:07.658219 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-e30d8e42-a0cf-4b1b-9c75-3ab8524b4d39 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:07.660201 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-e30d8e42-a0cf-4b1b-9c75-3ab8524b4d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:07.681555 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:07.702498 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c4ec5-f9dd-4c62-a236-a2c42cdeb086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:07.703241 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c4ec5-f9dd-4c62-a236-a2c42cdeb086 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:07.703453 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c4ec5-f9dd-4c62-a236-a2c42cdeb086 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:07.703722 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c4ec5-f9dd-4c62-a236-a2c42cdeb086 None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:07.704417 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c4ec5-f9dd-4c62-a236-a2c42cdeb086 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:07.704721 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c4ec5-f9dd-4c62-a236-a2c42cdeb086 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:07.706211 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c4ec5-f9dd-4c62-a236-a2c42cdeb086 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:07.718864 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 474/956] 10.222.0.44 () {56 vars in 1112 bytes} [Sun Sep 11 15:00:07 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:07.780568 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 483/957] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 15:00:08.051263 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c4ec5-f9dd-4c62-a236-a2c42cdeb086 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:08.165348 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c4ec5-f9dd-4c62-a236-a2c42cdeb086 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:08.184202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:08.204690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbe0e4-1964-43bb-aa48-66f2b9650f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:08.205398 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbe0e4-1964-43bb-aa48-66f2b9650f70 None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:08.205673 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbe0e4-1964-43bb-aa48-66f2b9650f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:08.205935 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbe0e4-1964-43bb-aa48-66f2b9650f70 None admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:08.206597 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cbe0e4-1964-43bb-aa48-66f2b9650f70 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:08.208065 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cbe0e4-1964-43bb-aa48-66f2b9650f70 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:08.210902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: WARNING keystone.server.flask.common [None req-35cbe0e4-1964-43bb-aa48-66f2b9650f70 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 11 15:00:08.231559 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.sql.core [None req-35cbe0e4-1964-43bb-aa48-66f2b9650f70 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 11 15:00:08.231559 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 11 15:00:08.231559 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [parameters: {'id': '2ae495bfa0bc47298e97aec9dc4d3dd3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 11 15:00:08.231559 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81684) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 11 15:00:08.248640 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-35cbe0e4-1964-43bb-aa48-66f2b9650f70 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2ae495bfa0bc47298e97aec9dc4d3dd3.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2ae495bfa0bc47298e97aec9dc4d3dd3. Sep 11 15:00:08.248640 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:08.248640 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 15:00:08.248640 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 15:00:08.248640 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 15:00:08.248640 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 15:00:08.248640 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 15:00:08.248640 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 15:00:08.248640 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 15:00:08.248640 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application conn.query(q) Sep 11 15:00:08.248640 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 15:00:08.248640 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 15:00:08.248640 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 15:00:08.249610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application result.read() Sep 11 15:00:08.249610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 15:00:08.249610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 15:00:08.249610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 15:00:08.249610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 15:00:08.249610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 15:00:08.249610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 15:00:08.249610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 15:00:08.249610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 15:00:08.249610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 11 15:00:08.249610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:08.249610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 11 15:00:08.249610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:08.249610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:08.249610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 11 15:00:08.249610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 11 15:00:08.249610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 11 15:00:08.249610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 11 15:00:08.250595 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 15:00:08.250595 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application next(self.gen) Sep 11 15:00:08.250595 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 11 15:00:08.250595 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application yield resource Sep 11 15:00:08.250595 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 15:00:08.250595 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application next(self.gen) Sep 11 15:00:08.250595 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 11 15:00:08.250595 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self.session.rollback() Sep 11 15:00:08.250595 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 11 15:00:08.250595 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self.force_reraise() Sep 11 15:00:08.250595 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 11 15:00:08.250595 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise self.value Sep 11 15:00:08.250595 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 11 15:00:08.250595 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 11 15:00:08.250595 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 11 15:00:08.250595 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application session.commit() Sep 11 15:00:08.250595 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 11 15:00:08.251519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 11 15:00:08.251519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 11 15:00:08.251519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._prepare_impl() Sep 11 15:00:08.251519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 11 15:00:08.251519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self.session.flush() Sep 11 15:00:08.251519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 11 15:00:08.251519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._flush(objects) Sep 11 15:00:08.251519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 11 15:00:08.251519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 11 15:00:08.251519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 11 15:00:08.251519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application compat.raise_( Sep 11 15:00:08.251519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 15:00:08.251519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception Sep 11 15:00:08.251519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 11 15:00:08.251519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application flush_context.execute() Sep 11 15:00:08.251519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 11 15:00:08.251519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rec.execute(self) Sep 11 15:00:08.252518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 11 15:00:08.252518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 11 15:00:08.252518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 11 15:00:08.252518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 11 15:00:08.252518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 11 15:00:08.252518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 11 15:00:08.252518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 11 15:00:08.252518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 11 15:00:08.252518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 11 15:00:08.252518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 11 15:00:08.252518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 11 15:00:08.252518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 11 15:00:08.252518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 11 15:00:08.252518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 11 15:00:08.252518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 11 15:00:08.253370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 11 15:00:08.253370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 15:00:08.253370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception Sep 11 15:00:08.253370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 15:00:08.253370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 15:00:08.253370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 15:00:08.253370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 15:00:08.253370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 15:00:08.253370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 15:00:08.253370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 15:00:08.253370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application conn.query(q) Sep 11 15:00:08.253370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 15:00:08.253370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 15:00:08.253370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 15:00:08.253370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application result.read() Sep 11 15:00:08.253370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 15:00:08.253370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 15:00:08.254270 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 15:00:08.254270 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 15:00:08.254270 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 15:00:08.254270 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 15:00:08.254270 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 15:00:08.254270 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 15:00:08.254270 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 11 15:00:08.254270 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 11 15:00:08.254270 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [parameters: {'id': '2ae495bfa0bc47298e97aec9dc4d3dd3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 11 15:00:08.254270 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 11 15:00:08.254270 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:08.254270 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 15:00:08.254270 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:08.254270 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:08.254270 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 11 15:00:08.255087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 11 15:00:08.255087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 11 15:00:08.255087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 11 15:00:08.255087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 11 15:00:08.255087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:08.255087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 15:00:08.255087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:08.255087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:08.255087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:08.255087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:08.255087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:08.255087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:08.255087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:08.255087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:08.255087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:08.255087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:08.255087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:08.256046 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:08.256046 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 11 15:00:08.256046 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 11 15:00:08.256046 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:08.256046 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:08.256046 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 11 15:00:08.256046 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 11 15:00:08.256046 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2ae495bfa0bc47298e97aec9dc4d3dd3. Sep 11 15:00:08.256046 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:08.260138 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 475/958] 10.222.0.44 () {60 vars in 1140 bytes} [Sun Sep 11 15:00:08 2022] POST /identity/v3/projects => generated 262 bytes in 209 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:08.266445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d8e42-a0cf-4b1b-9c75-3ab8524b4d39 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:08.386745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d8e42-a0cf-4b1b-9c75-3ab8524b4d39 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:08.424285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32220e6-eccd-49b0-8383-4a96e424e898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:08.424935 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32220e6-eccd-49b0-8383-4a96e424e898 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:08.425303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32220e6-eccd-49b0-8383-4a96e424e898 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:08.425576 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32220e6-eccd-49b0-8383-4a96e424e898 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:08.433300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32220e6-eccd-49b0-8383-4a96e424e898 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:08.434811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32220e6-eccd-49b0-8383-4a96e424e898 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:08.447188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-e32220e6-eccd-49b0-8383-4a96e424e898 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 15:00:08.447188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:08.447188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:08.447188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:08.447188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:08.447188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:08.447188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:08.447188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:08.447188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:08.447188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:08.447188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:08.447188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:08.447188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 15:00:08.447188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 15:00:08.447188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 15:00:08.448546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 15:00:08.448546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:08.448546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:08.448546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:08.448546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:08.448546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:08.448546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:08.448546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:08.448546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:08.448546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:08.448546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:08.448546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:08.448546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:08.448546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:08.448546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:08.448546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:08.448546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:08.449747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 15:00:08.449747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 15:00:08.449747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 15:00:08.449747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 15:00:08.449747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 15:00:08.449747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 15:00:08.449747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 15:00:08.449747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:08.449747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 484/959] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 15:00:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:08.452397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b273c-0c0c-4638-9d62-975877d55d02 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:08.452654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b273c-0c0c-4638-9d62-975877d55d02 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:08.452992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b273c-0c0c-4638-9d62-975877d55d02 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:08.455623 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32220e6-eccd-49b0-8383-4a96e424e898 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:08.463060 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 476/960] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:08 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:08.463430 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:08.471246 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac057bf5-ba36-4810-83a0-9f548d8a4101 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:08.471502 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac057bf5-ba36-4810-83a0-9f548d8a4101 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:08.472271 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac057bf5-ba36-4810-83a0-9f548d8a4101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:08.495065 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-ac057bf5-ba36-4810-83a0-9f548d8a4101 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:08.496645 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-ac057bf5-ba36-4810-83a0-9f548d8a4101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:08.607341 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32220e6-eccd-49b0-8383-4a96e424e898 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:08.629854 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 477/961] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:00:08.644965 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82282f49-ee5b-4202-93c3-8529ba827014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:08.645506 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82282f49-ee5b-4202-93c3-8529ba827014 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:08.645775 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82282f49-ee5b-4202-93c3-8529ba827014 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:08.646051 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82282f49-ee5b-4202-93c3-8529ba827014 None admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:08.646713 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82282f49-ee5b-4202-93c3-8529ba827014 None admin] RBAC: Adding query filter params (name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:08.647014 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82282f49-ee5b-4202-93c3-8529ba827014 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:08.648482 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82282f49-ee5b-4202-93c3-8529ba827014 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:08.668009 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 485/962] 10.222.0.44 () {56 vars in 1112 bytes} [Sun Sep 11 15:00:08 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:09.117007 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac057bf5-ba36-4810-83a0-9f548d8a4101 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:09.180885 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469ab54-7c9b-4dd8-9b8b-93313e999abf None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:09.182417 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469ab54-7c9b-4dd8-9b8b-93313e999abf None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:09.183160 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469ab54-7c9b-4dd8-9b8b-93313e999abf None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:09.186154 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 486/963] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:09.195943 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ce9d2-6e0a-4b8a-bce4-bb8fb3a10b56 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:09.196237 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ce9d2-6e0a-4b8a-bce4-bb8fb3a10b56 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:09.196570 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ce9d2-6e0a-4b8a-bce4-bb8fb3a10b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:09.217067 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-216ce9d2-6e0a-4b8a-bce4-bb8fb3a10b56 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:09.218319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-216ce9d2-6e0a-4b8a-bce4-bb8fb3a10b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:09.301791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac057bf5-ba36-4810-83a0-9f548d8a4101 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:09.304345 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:09.325909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:09.348935 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893a0f2-8b0c-4f29-a65e-3dedeb8902f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:09.349183 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 487/964] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:00:09.349953 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893a0f2-8b0c-4f29-a65e-3dedeb8902f1 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:09.350383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893a0f2-8b0c-4f29-a65e-3dedeb8902f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:09.350680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893a0f2-8b0c-4f29-a65e-3dedeb8902f1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:09.366439 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893a0f2-8b0c-4f29-a65e-3dedeb8902f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 15:00:09.371446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-e893a0f2-8b0c-4f29-a65e-3dedeb8902f1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 15:00:09.371446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:09.371446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:09.371446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:09.371446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:09.371446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:09.371446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:09.371446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:09.371446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:09.371446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:09.371446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:09.371446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:09.371446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 15:00:09.371446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 15:00:09.371446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 15:00:09.371446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 15:00:09.373152 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 15:00:09.373152 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 15:00:09.373152 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 15:00:09.373152 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 15:00:09.373152 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:09.373152 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:09.373152 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:09.373152 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:09.373152 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:09.373152 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:09.373152 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:09.373152 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:09.373152 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:09.373152 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:09.373152 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:09.373152 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:09.374775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:09.374775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:09.374775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:09.374775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:09.374775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 15:00:09.374775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 15:00:09.374775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 15:00:09.374775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 15:00:09.374775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 15:00:09.374775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 15:00:09.374775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 15:00:09.374775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:09.374775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 478/965] 10.222.0.44 () {56 vars in 1120 bytes} [Sun Sep 11 15:00:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:09.386365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ce9d2-6e0a-4b8a-bce4-bb8fb3a10b56 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:09.584107 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ce9d2-6e0a-4b8a-bce4-bb8fb3a10b56 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:09.651059 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ccaa8-d43e-4009-8292-6d899c4f4807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:09.652582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ccaa8-d43e-4009-8292-6d899c4f4807 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:09.653379 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ccaa8-d43e-4009-8292-6d899c4f4807 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:09.654120 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ccaa8-d43e-4009-8292-6d899c4f4807 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:09.655582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ccaa8-d43e-4009-8292-6d899c4f4807 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:09.656415 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ccaa8-d43e-4009-8292-6d899c4f4807 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:09.659281 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ccaa8-d43e-4009-8292-6d899c4f4807 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:09.675171 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 488/966] 10.222.0.44 () {56 vars in 1115 bytes} [Sun Sep 11 15:00:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:09.684398 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893a0f2-8b0c-4f29-a65e-3dedeb8902f1 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:09.815241 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ccaa8-d43e-4009-8292-6d899c4f4807 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:09.840015 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893a0f2-8b0c-4f29-a65e-3dedeb8902f1 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:09.902403 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f21cdb6-694a-4fda-9075-6e175a628efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:09.903354 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21cdb6-694a-4fda-9075-6e175a628efd None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:09.903916 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21cdb6-694a-4fda-9075-6e175a628efd None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:09.904355 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21cdb6-694a-4fda-9075-6e175a628efd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:09.914744 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:09.926360 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f21cdb6-694a-4fda-9075-6e175a628efd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:09.930458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f21cdb6-694a-4fda-9075-6e175a628efd None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:09.958902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-7f21cdb6-694a-4fda-9075-6e175a628efd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:00:09.958902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:09.958902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:09.958902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:09.958902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:09.958902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:09.958902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:09.958902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:09.958902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:09.958902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:09.958902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:09.958902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:09.958902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 15:00:09.958902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 15:00:09.958902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 15:00:09.958902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 15:00:09.961058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:09.961058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:09.961058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:09.961058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:09.961058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:09.961058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:09.961058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:09.961058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:09.961058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:09.961058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:09.961058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:09.961058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:09.961058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:09.961058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:09.961058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:09.961058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:09.961058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 15:00:09.963144 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 15:00:09.963144 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:00:09.963144 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:09.966161 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 479/967] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:00:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:09.972592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f21cdb6-694a-4fda-9075-6e175a628efd None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:10.002624 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ccaa8-d43e-4009-8292-6d899c4f4807 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:10.008745 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:10.064050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c188a-20a1-452f-abe7-a62b33323154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:10.064681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c188a-20a1-452f-abe7-a62b33323154 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:10.064975 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c188a-20a1-452f-abe7-a62b33323154 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:10.065264 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c188a-20a1-452f-abe7-a62b33323154 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:10.065863 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c188a-20a1-452f-abe7-a62b33323154 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:10.067256 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c188a-20a1-452f-abe7-a62b33323154 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:10.076783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-326c188a-20a1-452f-abe7-a62b33323154 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 15:00:10.076783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:10.076783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 15:00:10.076783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 15:00:10.076783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 15:00:10.076783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 15:00:10.076783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 15:00:10.076783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:00:10.076783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 15:00:10.076783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:10.076783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 15:00:10.076783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:10.076783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:10.076783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:10.076783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:10.076783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:10.076783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:10.078087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:10.078087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:10.078087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:10.078087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:10.078087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:10.078087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:10.078087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 15:00:10.078087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 15:00:10.078087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 15:00:10.078087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 15:00:10.078087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:10.078087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:10.078087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:10.078087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:10.078087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:10.078087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:10.079279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:10.079279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:10.079279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:10.079279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:10.079279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:10.079279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:10.079279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:10.079279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:10.079279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:10.079279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:10.079279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 15:00:10.079279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:00:10.079279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 15:00:10.079279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:10.079279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 489/968] 10.222.0.44 () {56 vars in 1126 bytes} [Sun Sep 11 15:00:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:10.080297 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44dd79-a73b-4915-b79b-e9a61e5a4bd9 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:10.080297 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44dd79-a73b-4915-b79b-e9a61e5a4bd9 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:10.080297 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44dd79-a73b-4915-b79b-e9a61e5a4bd9 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:10.085770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 490/969] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:10.085770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44dd79-a73b-4915-b79b-e9a61e5a4bd9 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:10.123858 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f21cdb6-694a-4fda-9075-6e175a628efd None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:10.164840 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1755288-ff72-46f1-aeb6-64b344b2e607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:10.165389 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1755288-ff72-46f1-aeb6-64b344b2e607 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:10.165669 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1755288-ff72-46f1-aeb6-64b344b2e607 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:10.165938 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1755288-ff72-46f1-aeb6-64b344b2e607 None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:10.166682 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1755288-ff72-46f1-aeb6-64b344b2e607 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:10.166995 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1755288-ff72-46f1-aeb6-64b344b2e607 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:10.168384 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1755288-ff72-46f1-aeb6-64b344b2e607 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:10.182406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 480/970] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 15:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:10.183886 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e427264-6ad3-4d6d-9b08-56a27a95df24 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:10.184173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e427264-6ad3-4d6d-9b08-56a27a95df24 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:10.184443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e427264-6ad3-4d6d-9b08-56a27a95df24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:10.204542 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-7e427264-6ad3-4d6d-9b08-56a27a95df24 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:10.206080 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-7e427264-6ad3-4d6d-9b08-56a27a95df24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:10.244970 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44dd79-a73b-4915-b79b-e9a61e5a4bd9 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:10.300714 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:10.336582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb766a40-cd74-4a17-a428-6cd7114e6b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:10.337687 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb766a40-cd74-4a17-a428-6cd7114e6b69 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:10.338172 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb766a40-cd74-4a17-a428-6cd7114e6b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:10.338617 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb766a40-cd74-4a17-a428-6cd7114e6b69 None admin] PATH_INFO: `/v3/services` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:10.339785 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb766a40-cd74-4a17-a428-6cd7114e6b69 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:10.340185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb766a40-cd74-4a17-a428-6cd7114e6b69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:10.343449 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb766a40-cd74-4a17-a428-6cd7114e6b69 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:10.361099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 491/971] 10.222.0.44 () {56 vars in 1122 bytes} [Sun Sep 11 15:00:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:10.362796 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb766a40-cd74-4a17-a428-6cd7114e6b69 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:10.370203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 481/972] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:00:10.371092 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e427264-6ad3-4d6d-9b08-56a27a95df24 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:10.393698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:10.626686 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb766a40-cd74-4a17-a428-6cd7114e6b69 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:10.637869 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e427264-6ad3-4d6d-9b08-56a27a95df24 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:10.717205 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabfef81-efed-490c-8697-f80ea225f30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:10.719070 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfef81-efed-490c-8697-f80ea225f30f None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:10.720014 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776161d-36cc-432b-bfae-2acacb4acebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:10.721063 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfef81-efed-490c-8697-f80ea225f30f None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:10.721590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfef81-efed-490c-8697-f80ea225f30f None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:10.722583 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776161d-36cc-432b-bfae-2acacb4acebf None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:10.723862 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabfef81-efed-490c-8697-f80ea225f30f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:10.724555 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776161d-36cc-432b-bfae-2acacb4acebf None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:10.725059 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776161d-36cc-432b-bfae-2acacb4acebf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:10.725670 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabfef81-efed-490c-8697-f80ea225f30f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:10.729233 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabfef81-efed-490c-8697-f80ea225f30f None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:10.747046 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 482/973] 10.222.0.44 () {56 vars in 1122 bytes} [Sun Sep 11 15:00:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:10.748468 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabfef81-efed-490c-8697-f80ea225f30f None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:10.749901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776161d-36cc-432b-bfae-2acacb4acebf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:10.753489 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776161d-36cc-432b-bfae-2acacb4acebf None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:10.778551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-0776161d-36cc-432b-bfae-2acacb4acebf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 15:00:10.778551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:10.778551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:10.778551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:10.778551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:10.778551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:10.778551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:10.778551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:10.778551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:10.778551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:10.778551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:10.778551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:10.778551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 15:00:10.778551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 15:00:10.778551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 15:00:10.778551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 15:00:10.780405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:10.780405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:10.780405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 15:00:10.780405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 15:00:10.780405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 15:00:10.780405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 15:00:10.780405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:10.780405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:10.780405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:10.780405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:10.780405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:10.780405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:10.780405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:10.780405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:10.780405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:10.780405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:10.780405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:10.780405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:10.782962 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:10.782962 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:10.782962 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 15:00:10.782962 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 15:00:10.782962 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 15:00:10.782962 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 15:00:10.782962 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 15:00:10.782962 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 15:00:10.782962 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 15:00:10.782962 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:10.782962 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 492/974] 10.222.0.44 () {56 vars in 1117 bytes} [Sun Sep 11 15:00:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 420 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:10.782962 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776161d-36cc-432b-bfae-2acacb4acebf None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:10.867719 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:10.874686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:10.931673 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabfef81-efed-490c-8697-f80ea225f30f None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:10.962220 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776161d-36cc-432b-bfae-2acacb4acebf None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:10.989794 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e834e-1b3d-4c23-8883-5aedaa12b14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:10.990551 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e834e-1b3d-4c23-8883-5aedaa12b14f None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:10.990991 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e834e-1b3d-4c23-8883-5aedaa12b14f None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:10.991364 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e834e-1b3d-4c23-8883-5aedaa12b14f None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:10.992414 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e834e-1b3d-4c23-8883-5aedaa12b14f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:10.995067 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e834e-1b3d-4c23-8883-5aedaa12b14f None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:11.018230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 483/975] 10.222.0.44 () {60 vars in 1141 bytes} [Sun Sep 11 15:00:10 2022] POST /identity/v3/services => generated 242 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:00:11.019523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e834e-1b3d-4c23-8883-5aedaa12b14f None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:11.039891 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f5c177-92c7-451b-b583-26c51670d27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:11.041070 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5c177-92c7-451b-b583-26c51670d27a None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:11.042455 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5c177-92c7-451b-b583-26c51670d27a None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:11.043005 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5c177-92c7-451b-b583-26c51670d27a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:11.044075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5c177-92c7-451b-b583-26c51670d27a None admin] RBAC: Adding query filter params (interface=admin, service_id=a123fbac018f43e7a264df85406f2166, region_id=RegionOne) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:11.044608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5c177-92c7-451b-b583-26c51670d27a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:11.047021 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5c177-92c7-451b-b583-26c51670d27a None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:11.056056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 493/976] 10.222.0.44 () {56 vars in 1249 bytes} [Sun Sep 11 15:00:10 2022] GET /identity/v3/endpoints?service_id=a123fbac018f43e7a264df85406f2166&interface=admin®ion_id=RegionOne => generated 193 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:11.062937 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f5c177-92c7-451b-b583-26c51670d27a None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:11.175130 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e834e-1b3d-4c23-8883-5aedaa12b14f None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:11.209973 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:11.223390 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37d09d-6e23-418f-8661-69154f200699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:11.224328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37d09d-6e23-418f-8661-69154f200699 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:11.224757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37d09d-6e23-418f-8661-69154f200699 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:11.225219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37d09d-6e23-418f-8661-69154f200699 None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:11.226428 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37d09d-6e23-418f-8661-69154f200699 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:11.226863 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37d09d-6e23-418f-8661-69154f200699 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:11.229402 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37d09d-6e23-418f-8661-69154f200699 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:11.231606 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f5c177-92c7-451b-b583-26c51670d27a None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:11.234249 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:11.263364 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 484/977] 10.222.0.44 () {56 vars in 1149 bytes} [Sun Sep 11 15:00:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:11.265116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05f007-f58e-46e9-b48f-323c58e95d53 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:11.265592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05f007-f58e-46e9-b48f-323c58e95d53 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:11.266016 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05f007-f58e-46e9-b48f-323c58e95d53 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:11.268014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 485/978] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:11.270429 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05f007-f58e-46e9-b48f-323c58e95d53 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:11.286948 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af2e99-5ada-4847-8b4f-8fcfe5f9cff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:11.287806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af2e99-5ada-4847-8b4f-8fcfe5f9cff1 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:11.287954 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af2e99-5ada-4847-8b4f-8fcfe5f9cff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:11.288102 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af2e99-5ada-4847-8b4f-8fcfe5f9cff1 None admin] PATH_INFO: `/v3/services` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:11.288848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40af2e99-5ada-4847-8b4f-8fcfe5f9cff1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:11.290237 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40af2e99-5ada-4847-8b4f-8fcfe5f9cff1 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:11.298879 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 494/979] 10.222.0.44 () {56 vars in 1087 bytes} [Sun Sep 11 15:00:11 2022] GET /identity/v3/services => generated 1727 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:00:11.304161 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec3845-087b-4061-a00f-6669758c1096 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:11.304669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec3845-087b-4061-a00f-6669758c1096 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:11.305260 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec3845-087b-4061-a00f-6669758c1096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:11.347807 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-65ec3845-087b-4061-a00f-6669758c1096 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:11.350121 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-65ec3845-087b-4061-a00f-6669758c1096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:11.421484 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05f007-f58e-46e9-b48f-323c58e95d53 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:11.467399 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd9923-7980-4a17-89fd-24e6d3bfe792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:11.468432 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd9923-7980-4a17-89fd-24e6d3bfe792 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:11.468928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd9923-7980-4a17-89fd-24e6d3bfe792 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:11.469467 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd9923-7980-4a17-89fd-24e6d3bfe792 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:11.501674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 495/980] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:11 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:00:11.571443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:11.594764 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd9923-7980-4a17-89fd-24e6d3bfe792 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:11.597598 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd9923-7980-4a17-89fd-24e6d3bfe792 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:11.767667 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 486/981] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 15:00:11 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 15:00:11.774132 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec3845-087b-4061-a00f-6669758c1096 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:11.789285 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd9923-7980-4a17-89fd-24e6d3bfe792 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:11.899307 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:11.909068 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec3845-087b-4061-a00f-6669758c1096 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:11.925355 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:11.948941 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd9923-7980-4a17-89fd-24e6d3bfe792 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:11.956633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5eaf08-b4f0-4216-9452-b9ef2ba0c071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:11.957595 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5eaf08-b4f0-4216-9452-b9ef2ba0c071 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:11.958083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5eaf08-b4f0-4216-9452-b9ef2ba0c071 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:11.958581 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5eaf08-b4f0-4216-9452-b9ef2ba0c071 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:11.974826 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5eaf08-b4f0-4216-9452-b9ef2ba0c071 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:11.977047 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5eaf08-b4f0-4216-9452-b9ef2ba0c071 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:11.998890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-ea5eaf08-b4f0-4216-9452-b9ef2ba0c071 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:00:11.998890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:11.998890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:11.998890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:11.998890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:11.998890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:11.998890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:11.998890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:11.998890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:11.998890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:11.998890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:11.998890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:11.998890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 15:00:11.998890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 15:00:11.998890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 15:00:11.998890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 15:00:12.000070 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:12.000070 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:12.000070 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:12.000070 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:12.000070 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:12.000070 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:12.000070 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:12.000070 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:12.000070 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:12.000070 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:12.000070 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:12.000070 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:12.000070 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:12.000070 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:12.000070 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:12.000070 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:12.000070 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 15:00:12.001085 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 15:00:12.001085 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:00:12.001085 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:12.004782 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8763b-847e-48de-b952-3ee128a94d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:12.005470 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 496/982] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:00:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 15:00:12.006204 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8763b-847e-48de-b952-3ee128a94d5d None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:12.006563 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5eaf08-b4f0-4216-9452-b9ef2ba0c071 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:12.007139 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8763b-847e-48de-b952-3ee128a94d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:12.007630 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8763b-847e-48de-b952-3ee128a94d5d None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:12.008657 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b8763b-847e-48de-b952-3ee128a94d5d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:12.012237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b8763b-847e-48de-b952-3ee128a94d5d None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:12.041601 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.sql.core [None req-a6b8763b-847e-48de-b952-3ee128a94d5d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 11 15:00:12.041601 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 11 15:00:12.041601 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [parameters: {'user_id': '255e3de565984622a4766754d38f0bb1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 11 15:00:12.041601 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81684) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 11 15:00:12.058972 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-a6b8763b-847e-48de-b952-3ee128a94d5d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 11 15:00:12.058972 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:12.058972 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 15:00:12.058972 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 15:00:12.058972 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 15:00:12.058972 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 15:00:12.058972 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 15:00:12.058972 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 15:00:12.058972 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 15:00:12.058972 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application conn.query(q) Sep 11 15:00:12.058972 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 15:00:12.058972 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 15:00:12.058972 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 15:00:12.058972 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application result.read() Sep 11 15:00:12.058972 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 15:00:12.060701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 15:00:12.060701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 15:00:12.060701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 15:00:12.060701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 15:00:12.060701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 15:00:12.060701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 15:00:12.060701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 15:00:12.060701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 11 15:00:12.060701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:12.060701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 11 15:00:12.060701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:12.060701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:12.060701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 11 15:00:12.060701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 11 15:00:12.060701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 11 15:00:12.060701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 11 15:00:12.060701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 15:00:12.060701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application next(self.gen) Sep 11 15:00:12.062592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 11 15:00:12.062592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application yield resource Sep 11 15:00:12.062592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 15:00:12.062592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application next(self.gen) Sep 11 15:00:12.062592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 11 15:00:12.062592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self.session.rollback() Sep 11 15:00:12.062592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 11 15:00:12.062592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self.force_reraise() Sep 11 15:00:12.062592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 11 15:00:12.062592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise self.value Sep 11 15:00:12.062592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 11 15:00:12.062592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 11 15:00:12.062592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 11 15:00:12.062592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application session.commit() Sep 11 15:00:12.062592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 11 15:00:12.062592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 11 15:00:12.062592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 11 15:00:12.066292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._prepare_impl() Sep 11 15:00:12.066292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 11 15:00:12.066292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self.session.flush() Sep 11 15:00:12.066292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 11 15:00:12.066292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._flush(objects) Sep 11 15:00:12.066292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 11 15:00:12.066292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 11 15:00:12.066292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 11 15:00:12.066292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application compat.raise_( Sep 11 15:00:12.066292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 15:00:12.066292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception Sep 11 15:00:12.066292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 11 15:00:12.066292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application flush_context.execute() Sep 11 15:00:12.066292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 11 15:00:12.066292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rec.execute(self) Sep 11 15:00:12.066292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 11 15:00:12.066292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 11 15:00:12.069855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 11 15:00:12.069855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 11 15:00:12.069855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 11 15:00:12.069855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 11 15:00:12.069855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 11 15:00:12.069855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 11 15:00:12.069855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 11 15:00:12.069855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 11 15:00:12.069855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 11 15:00:12.069855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 11 15:00:12.069855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 11 15:00:12.069855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 11 15:00:12.069855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 11 15:00:12.069855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 11 15:00:12.069855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 15:00:12.069855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception Sep 11 15:00:12.073459 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 15:00:12.073459 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 15:00:12.073459 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 15:00:12.073459 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 15:00:12.073459 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 15:00:12.073459 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 15:00:12.073459 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 15:00:12.073459 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application conn.query(q) Sep 11 15:00:12.073459 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 15:00:12.073459 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 15:00:12.073459 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 15:00:12.073459 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application result.read() Sep 11 15:00:12.073459 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 15:00:12.073459 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 15:00:12.073459 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 15:00:12.073459 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 15:00:12.076579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 15:00:12.076579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 15:00:12.076579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 15:00:12.076579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 15:00:12.076579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 11 15:00:12.076579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 11 15:00:12.076579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [parameters: {'user_id': '255e3de565984622a4766754d38f0bb1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 11 15:00:12.076579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 11 15:00:12.076579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:12.076579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 15:00:12.076579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:12.076579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:12.076579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:12.076579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:12.076579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:12.081125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:12.081125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:12.081125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:12.081125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:12.081125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:12.081125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:12.081125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:12.081125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 11 15:00:12.081125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 11 15:00:12.081125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:12.081125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:12.081125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 15:00:12.081125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 15:00:12.081125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 15:00:12.081125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 15:00:12.081125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 11 15:00:12.081125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 11 15:00:12.084581 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 11 15:00:12.084581 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 11 15:00:12.084581 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 11 15:00:12.084581 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 11 15:00:12.084581 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 11 15:00:12.084581 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:12.084581 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 487/983] 10.222.0.44 () {60 vars in 1131 bytes} [Sun Sep 11 15:00:11 2022] POST /identity/v3/users => generated 160 bytes in 280 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 11 15:00:12.084581 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8763b-847e-48de-b952-3ee128a94d5d None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:12.184481 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5eaf08-b4f0-4216-9452-b9ef2ba0c071 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:12.220886 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8763b-847e-48de-b952-3ee128a94d5d None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:12.232509 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b2cad-84b0-4292-85e6-d55f3cea16a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:12.233532 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b2cad-84b0-4292-85e6-d55f3cea16a4 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:12.234013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b2cad-84b0-4292-85e6-d55f3cea16a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:12.234505 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b2cad-84b0-4292-85e6-d55f3cea16a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:12.236155 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b2cad-84b0-4292-85e6-d55f3cea16a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:12.236774 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b2cad-84b0-4292-85e6-d55f3cea16a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:12.238418 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b2cad-84b0-4292-85e6-d55f3cea16a4 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:12.257808 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 497/984] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 15:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:12.266939 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b2cad-84b0-4292-85e6-d55f3cea16a4 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:12.280857 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:12.282067 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67596a5e-df9c-4ca5-aebb-c1f848ef8478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:12.282774 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67596a5e-df9c-4ca5-aebb-c1f848ef8478 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:12.283053 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67596a5e-df9c-4ca5-aebb-c1f848ef8478 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:12.283888 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67596a5e-df9c-4ca5-aebb-c1f848ef8478 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:12.295155 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67596a5e-df9c-4ca5-aebb-c1f848ef8478 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:12.296761 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67596a5e-df9c-4ca5-aebb-c1f848ef8478 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:12.313284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-67596a5e-df9c-4ca5-aebb-c1f848ef8478 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 15:00:12.313284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:12.313284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:12.313284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:12.313284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:12.313284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:12.313284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:12.313284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:12.313284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:12.313284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:12.313284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:12.313284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:12.313284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 15:00:12.313284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 15:00:12.313284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 15:00:12.313284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 15:00:12.314406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:12.314406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:12.314406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 15:00:12.314406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 15:00:12.314406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 15:00:12.314406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 15:00:12.314406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:12.314406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:12.314406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:12.314406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:12.314406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:12.314406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:12.314406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:12.314406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:12.314406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:12.314406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:12.314406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:12.314406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:12.316337 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:12.316337 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:12.316337 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 15:00:12.316337 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 15:00:12.316337 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 15:00:12.316337 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 15:00:12.316337 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 15:00:12.316337 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 15:00:12.316337 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 15:00:12.316337 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:12.325396 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 488/985] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 15:00:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:12.330913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67596a5e-df9c-4ca5-aebb-c1f848ef8478 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:12.353442 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:12.398511 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b2cad-84b0-4292-85e6-d55f3cea16a4 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:12.444813 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a332c51-ee14-4935-9465-94462f360a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:12.445881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a332c51-ee14-4935-9465-94462f360a88 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:12.446163 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a332c51-ee14-4935-9465-94462f360a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:12.446434 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a332c51-ee14-4935-9465-94462f360a88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:12.456555 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a332c51-ee14-4935-9465-94462f360a88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:12.458652 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a332c51-ee14-4935-9465-94462f360a88 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:12.475786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-7a332c51-ee14-4935-9465-94462f360a88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 15:00:12.475786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:12.475786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:12.475786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:12.475786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:12.475786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:12.475786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:12.475786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:12.475786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:12.475786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:12.475786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:12.475786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:12.475786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 15:00:12.475786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 15:00:12.475786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 15:00:12.477062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 15:00:12.477062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:12.477062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:12.477062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:12.477062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:12.477062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:12.477062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:12.477062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:12.477062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:12.477062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:12.477062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:12.477062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:12.477062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:12.477062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:12.477062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:12.477062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:12.477062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:12.478899 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 15:00:12.478899 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 15:00:12.478899 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 15:00:12.478899 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 15:00:12.478899 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 15:00:12.478899 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 15:00:12.478899 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 15:00:12.478899 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:12.482364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 498/986] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 15:00:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 15:00:12.483653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a332c51-ee14-4935-9465-94462f360a88 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:12.503390 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67596a5e-df9c-4ca5-aebb-c1f848ef8478 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:12.561221 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ebf288-cb52-47f1-9f3c-c3c35d99e938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:12.562481 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebf288-cb52-47f1-9f3c-c3c35d99e938 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:12.563110 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebf288-cb52-47f1-9f3c-c3c35d99e938 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:12.563605 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebf288-cb52-47f1-9f3c-c3c35d99e938 None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:12.565022 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ebf288-cb52-47f1-9f3c-c3c35d99e938 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:12.565593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ebf288-cb52-47f1-9f3c-c3c35d99e938 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:12.567642 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ebf288-cb52-47f1-9f3c-c3c35d99e938 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:12.576988 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:12.620831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a332c51-ee14-4935-9465-94462f360a88 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:12.633802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 489/987] 10.222.0.44 () {56 vars in 1101 bytes} [Sun Sep 11 15:00:12 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:12.662060 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29315db7-dbe4-4ae9-bb85-1790ac851170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:12.662702 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29315db7-dbe4-4ae9-bb85-1790ac851170 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:12.662976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29315db7-dbe4-4ae9-bb85-1790ac851170 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:12.663258 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29315db7-dbe4-4ae9-bb85-1790ac851170 None admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:12.663962 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29315db7-dbe4-4ae9-bb85-1790ac851170 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:12.664270 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29315db7-dbe4-4ae9-bb85-1790ac851170 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:12.665832 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29315db7-dbe4-4ae9-bb85-1790ac851170 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:12.681959 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 499/988] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 15:00:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:12.688297 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ebf288-cb52-47f1-9f3c-c3c35d99e938 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:12.770538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:12.829219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ebf288-cb52-47f1-9f3c-c3c35d99e938 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:12.892553 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04dd42-ad56-467b-a235-dcd2cfb73235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:12.894213 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04dd42-ad56-467b-a235-dcd2cfb73235 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:12.895035 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04dd42-ad56-467b-a235-dcd2cfb73235 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:12.895790 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04dd42-ad56-467b-a235-dcd2cfb73235 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:12.897518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04dd42-ad56-467b-a235-dcd2cfb73235 None admin] RBAC: Adding query filter params (role.id=92fb89d658dc4d4788cef145280a11f7, scope.project.id=70ac515938a24c45ae6302dbac46a1ab, user.id=3fada78ceaeb4f3bad77c516f6dfcfd4) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:12.898452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04dd42-ad56-467b-a235-dcd2cfb73235 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:12.901236 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04dd42-ad56-467b-a235-dcd2cfb73235 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:12.912626 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 490/989] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 15:00:12 2022] GET /identity/v3/role_assignments?user.id=3fada78ceaeb4f3bad77c516f6dfcfd4&scope.project.id=70ac515938a24c45ae6302dbac46a1ab&role.id=92fb89d658dc4d4788cef145280a11f7 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:13.316762 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a2c88-2820-4650-b73e-becfd9e931f7 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:13.317037 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a2c88-2820-4650-b73e-becfd9e931f7 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:13.317373 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a2c88-2820-4650-b73e-becfd9e931f7 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:13.319040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 500/990] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:13.326106 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be759a-7630-4ed5-bf2d-c7e1642ec0e3 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:13.326643 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be759a-7630-4ed5-bf2d-c7e1642ec0e3 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:13.326967 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be759a-7630-4ed5-bf2d-c7e1642ec0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:13.356840 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-66be759a-7630-4ed5-bf2d-c7e1642ec0e3 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:13.358720 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-66be759a-7630-4ed5-bf2d-c7e1642ec0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:13.437181 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 491/991] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:13 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:00:13.686544 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f23e50-4c0d-4343-b2d9-84b76cacd2d4 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:13.687085 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f23e50-4c0d-4343-b2d9-84b76cacd2d4 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:13.687085 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f23e50-4c0d-4343-b2d9-84b76cacd2d4 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:13.688265 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 501/992] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:13.694203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9980cb4-8459-42fd-9292-2bc52ae05a4f None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:13.694502 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9980cb4-8459-42fd-9292-2bc52ae05a4f None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:13.694707 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9980cb4-8459-42fd-9292-2bc52ae05a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:13.699135 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f23e50-4c0d-4343-b2d9-84b76cacd2d4 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:13.717835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-e9980cb4-8459-42fd-9292-2bc52ae05a4f None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:13.719018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-e9980cb4-8459-42fd-9292-2bc52ae05a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:13.790565 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:13.805232 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 492/993] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:13 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:00:13.828572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f23e50-4c0d-4343-b2d9-84b76cacd2d4 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:13.860371 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:13.870188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f640e-b209-4bf7-84b2-78c7c1d71eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:13.871031 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f640e-b209-4bf7-84b2-78c7c1d71eba None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:13.871322 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f640e-b209-4bf7-84b2-78c7c1d71eba None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:13.871581 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f640e-b209-4bf7-84b2-78c7c1d71eba None admin] PATH_INFO: `/v3/services/compute` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:13.872169 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f640e-b209-4bf7-84b2-78c7c1d71eba None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:13.873602 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f640e-b209-4bf7-84b2-78c7c1d71eba None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:13.884331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-622f640e-b209-4bf7-84b2-78c7c1d71eba None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 15:00:13.884331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:13.884331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 15:00:13.884331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 15:00:13.884331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 15:00:13.884331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 15:00:13.884331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 15:00:13.884331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:00:13.884331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 15:00:13.884331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:13.884331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 15:00:13.884331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:13.884331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:13.884331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:13.884331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:13.884331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:13.884331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:13.890852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:13.890852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:13.890852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:13.890852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:13.890852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:13.890852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:13.890852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 15:00:13.890852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 15:00:13.890852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 15:00:13.890852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 15:00:13.890852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:13.890852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:13.890852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:13.890852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:13.890852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:13.890852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:13.891716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:13.891716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:13.891716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:13.891716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:13.891716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:13.891716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:13.891716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:13.891716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:13.891716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:13.891716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:13.891716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 15:00:13.891716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:00:13.891716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 15:00:13.891716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:13.891716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 502/994] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 15:00:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:13.892512 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9980cb4-8459-42fd-9292-2bc52ae05a4f None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:14.013242 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9980cb4-8459-42fd-9292-2bc52ae05a4f None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:14.053494 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f8372-2ce1-4f44-b0c6-3975dd8cf82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:14.054114 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f8372-2ce1-4f44-b0c6-3975dd8cf82e None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:14.054392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f8372-2ce1-4f44-b0c6-3975dd8cf82e None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:14.054656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f8372-2ce1-4f44-b0c6-3975dd8cf82e None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:14.055291 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f8372-2ce1-4f44-b0c6-3975dd8cf82e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:14.055584 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f8372-2ce1-4f44-b0c6-3975dd8cf82e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:14.056998 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f8372-2ce1-4f44-b0c6-3975dd8cf82e None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:14.062547 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 493/995] 10.222.0.44 () {56 vars in 1112 bytes} [Sun Sep 11 15:00:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:14.066083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f640e-b209-4bf7-84b2-78c7c1d71eba None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:14.067989 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f8372-2ce1-4f44-b0c6-3975dd8cf82e None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:14.139137 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:14.196651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f8372-2ce1-4f44-b0c6-3975dd8cf82e None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:14.203996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f640e-b209-4bf7-84b2-78c7c1d71eba None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:14.249046 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a1f0d-aca2-4131-a35d-90d2126d93b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:14.249817 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a1f0d-aca2-4131-a35d-90d2126d93b2 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:14.250150 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a1f0d-aca2-4131-a35d-90d2126d93b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:14.250547 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a1f0d-aca2-4131-a35d-90d2126d93b2 None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:14.251201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a1f0d-aca2-4131-a35d-90d2126d93b2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:14.251782 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a1f0d-aca2-4131-a35d-90d2126d93b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:14.253018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a1f0d-aca2-4131-a35d-90d2126d93b2 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:14.261903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 494/996] 10.222.0.44 () {56 vars in 1112 bytes} [Sun Sep 11 15:00:14 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:14.262434 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a177051-4da3-4fd6-acee-968cff1f45f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:14.263235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a177051-4da3-4fd6-acee-968cff1f45f5 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:14.263683 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a177051-4da3-4fd6-acee-968cff1f45f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:14.264010 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a177051-4da3-4fd6-acee-968cff1f45f5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:14.264720 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a177051-4da3-4fd6-acee-968cff1f45f5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:14.266298 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a177051-4da3-4fd6-acee-968cff1f45f5 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:14.271230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a1f0d-aca2-4131-a35d-90d2126d93b2 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:14.281721 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-3a177051-4da3-4fd6-acee-968cff1f45f5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 15:00:14.281721 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:14.281721 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 15:00:14.281721 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 15:00:14.281721 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 15:00:14.281721 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 15:00:14.281721 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 15:00:14.281721 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:00:14.281721 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 15:00:14.281721 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:14.281721 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 15:00:14.281721 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:14.281721 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:14.281721 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:14.281721 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:14.281721 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:14.281721 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:14.282770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:14.282770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:14.282770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:14.282770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:14.282770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:14.282770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:14.282770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 15:00:14.282770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 15:00:14.282770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 15:00:14.282770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 15:00:14.282770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:14.282770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:14.282770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:14.282770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:14.282770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:14.282770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:14.285993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:14.285993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:14.285993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:14.285993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:14.285993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:14.285993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:14.285993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:14.285993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:14.285993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:14.285993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:14.285993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 15:00:14.285993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:00:14.285993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 15:00:14.285993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:14.285993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 503/997] 10.222.0.44 () {56 vars in 1126 bytes} [Sun Sep 11 15:00:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:14.293504 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a177051-4da3-4fd6-acee-968cff1f45f5 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:14.379202 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:14.401421 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a1f0d-aca2-4131-a35d-90d2126d93b2 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:14.427174 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:14.456047 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a02d8e-60e8-41da-81c4-1cc9d8051d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:14.457035 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a02d8e-60e8-41da-81c4-1cc9d8051d7d None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:14.457521 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a02d8e-60e8-41da-81c4-1cc9d8051d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:14.457976 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a02d8e-60e8-41da-81c4-1cc9d8051d7d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:14.459189 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a02d8e-60e8-41da-81c4-1cc9d8051d7d None admin] RBAC: Adding query filter params (interface=public, service_id=af92612ae03443ae875424c9920ab4ed, region_id=RegionOne) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:14.459701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a02d8e-60e8-41da-81c4-1cc9d8051d7d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:14.462127 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a02d8e-60e8-41da-81c4-1cc9d8051d7d None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:14.466427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a177051-4da3-4fd6-acee-968cff1f45f5 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:14.471736 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 495/998] 10.222.0.44 () {56 vars in 1251 bytes} [Sun Sep 11 15:00:14 2022] GET /identity/v3/endpoints?service_id=af92612ae03443ae875424c9920ab4ed&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:14.478328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a02d8e-60e8-41da-81c4-1cc9d8051d7d None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:14.524725 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2820326-790e-44ff-b978-3aa0fa06eaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:14.525856 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2820326-790e-44ff-b978-3aa0fa06eaf4 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:14.526335 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2820326-790e-44ff-b978-3aa0fa06eaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:14.526826 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2820326-790e-44ff-b978-3aa0fa06eaf4 None admin] PATH_INFO: `/v3/services` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:14.527956 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2820326-790e-44ff-b978-3aa0fa06eaf4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:14.528473 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2820326-790e-44ff-b978-3aa0fa06eaf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:14.531107 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2820326-790e-44ff-b978-3aa0fa06eaf4 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:14.548373 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 504/999] 10.222.0.44 () {56 vars in 1122 bytes} [Sun Sep 11 15:00:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:14.550292 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2820326-790e-44ff-b978-3aa0fa06eaf4 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:14.609019 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a02d8e-60e8-41da-81c4-1cc9d8051d7d None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:14.647998 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cf5d8-5afd-4426-aae9-f9cba9a9d6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:14.648621 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cf5d8-5afd-4426-aae9-f9cba9a9d6e0 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:14.648896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cf5d8-5afd-4426-aae9-f9cba9a9d6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:14.649238 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cf5d8-5afd-4426-aae9-f9cba9a9d6e0 None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:14.649858 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0cf5d8-5afd-4426-aae9-f9cba9a9d6e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:14.651387 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0cf5d8-5afd-4426-aae9-f9cba9a9d6e0 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:14.658378 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 496/1000] 10.222.0.44 () {56 vars in 1087 bytes} [Sun Sep 11 15:00:14 2022] GET /identity/v3/services => generated 1727 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:00:14.755701 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2820326-790e-44ff-b978-3aa0fa06eaf4 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:14.799630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:14.829867 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6dfb7-df1b-4e81-93af-401e7c129f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:14.831011 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6dfb7-df1b-4e81-93af-401e7c129f77 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:14.831571 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6dfb7-df1b-4e81-93af-401e7c129f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:14.832119 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6dfb7-df1b-4e81-93af-401e7c129f77 None admin] PATH_INFO: `/v3/services` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:14.833391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db6dfb7-df1b-4e81-93af-401e7c129f77 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:14.833972 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db6dfb7-df1b-4e81-93af-401e7c129f77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:14.836786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db6dfb7-df1b-4e81-93af-401e7c129f77 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:14.853252 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 505/1001] 10.222.0.44 () {56 vars in 1122 bytes} [Sun Sep 11 15:00:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:14.857951 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cf5d8-5afd-4426-aae9-f9cba9a9d6e0 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:14.938581 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:15.036385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cf5d8-5afd-4426-aae9-f9cba9a9d6e0 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:15.094681 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847b493-fa75-450e-9a57-88fc7d7904d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:15.096011 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847b493-fa75-450e-9a57-88fc7d7904d8 None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:15.096492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847b493-fa75-450e-9a57-88fc7d7904d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:15.096962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847b493-fa75-450e-9a57-88fc7d7904d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:15.098323 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4847b493-fa75-450e-9a57-88fc7d7904d8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:15.100835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4847b493-fa75-450e-9a57-88fc7d7904d8 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:15.137166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 497/1002] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:00:14 2022] POST /identity/v3/endpoints => generated 325 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:00:15.224837 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fee5e7-9bbf-4ca0-a99c-74b12c3ee942 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:15.225384 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fee5e7-9bbf-4ca0-a99c-74b12c3ee942 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:15.225384 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fee5e7-9bbf-4ca0-a99c-74b12c3ee942 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:15.227086 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 506/1003] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 15:00:15.233100 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473206fa-41dd-456d-88bd-94c59405aa3c None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:15.233405 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473206fa-41dd-456d-88bd-94c59405aa3c None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:15.233736 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473206fa-41dd-456d-88bd-94c59405aa3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:15.257625 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-473206fa-41dd-456d-88bd-94c59405aa3c None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:15.260372 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-473206fa-41dd-456d-88bd-94c59405aa3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:15.384756 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 498/1004] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:15 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:00:15.855914 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fee5e7-9bbf-4ca0-a99c-74b12c3ee942 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:16.043929 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fee5e7-9bbf-4ca0-a99c-74b12c3ee942 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:16.107885 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a0a6c-7bfd-4a72-8b79-fb43049ee495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:16.108817 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a0a6c-7bfd-4a72-8b79-fb43049ee495 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:16.109341 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a0a6c-7bfd-4a72-8b79-fb43049ee495 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:16.109782 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a0a6c-7bfd-4a72-8b79-fb43049ee495 None admin] PATH_INFO: `/v3/services/image` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:16.110715 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38a0a6c-7bfd-4a72-8b79-fb43049ee495 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:16.113216 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38a0a6c-7bfd-4a72-8b79-fb43049ee495 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:16.130372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-a38a0a6c-7bfd-4a72-8b79-fb43049ee495 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 15:00:16.130372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:16.130372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 15:00:16.130372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 15:00:16.130372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 15:00:16.130372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 15:00:16.130372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 15:00:16.130372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:00:16.130372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 15:00:16.130372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:16.130372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 15:00:16.130372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:16.130372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:16.130372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:16.130372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:16.130372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:16.130372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:16.132196 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:16.132196 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:16.132196 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:16.132196 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:16.132196 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:16.132196 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:16.132196 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 15:00:16.132196 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 15:00:16.132196 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 15:00:16.132196 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 15:00:16.132196 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:16.132196 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:16.132196 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:16.132196 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:16.132196 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:16.132196 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:16.135199 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:16.135199 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:16.135199 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:16.135199 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:16.135199 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:16.135199 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:16.135199 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:16.135199 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:16.135199 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:16.135199 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:16.135199 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 15:00:16.135199 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:00:16.135199 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 15:00:16.135199 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:16.135199 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 507/1005] 10.222.0.44 () {56 vars in 1105 bytes} [Sun Sep 11 15:00:15 2022] GET /identity/v3/services/image => generated 86 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:16.142673 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473206fa-41dd-456d-88bd-94c59405aa3c None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:16.196245 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:16.321912 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473206fa-41dd-456d-88bd-94c59405aa3c None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:16.379399 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c9c5d-a7a8-40fc-8a3c-96852bfd7d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:16.379917 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c9c5d-a7a8-40fc-8a3c-96852bfd7d7b None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:16.380180 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c9c5d-a7a8-40fc-8a3c-96852bfd7d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:16.380456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c9c5d-a7a8-40fc-8a3c-96852bfd7d7b None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:16.381089 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468c9c5d-a7a8-40fc-8a3c-96852bfd7d7b None admin] RBAC: Adding query filter params (name=image) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:16.381417 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468c9c5d-a7a8-40fc-8a3c-96852bfd7d7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:16.382795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468c9c5d-a7a8-40fc-8a3c-96852bfd7d7b None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:16.388628 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 499/1006] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:00:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:16.394820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a0a6c-7bfd-4a72-8b79-fb43049ee495 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:16.479386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:16.568457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a0a6c-7bfd-4a72-8b79-fb43049ee495 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:16.635937 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988f57f-ac7d-4929-bbcf-7d8a761ad5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:16.636836 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988f57f-ac7d-4929-bbcf-7d8a761ad5c4 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:16.637388 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988f57f-ac7d-4929-bbcf-7d8a761ad5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:16.637851 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988f57f-ac7d-4929-bbcf-7d8a761ad5c4 None admin] PATH_INFO: `/v3/services` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:16.638968 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4988f57f-ac7d-4929-bbcf-7d8a761ad5c4 None admin] RBAC: Adding query filter params (type=image) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:16.639502 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4988f57f-ac7d-4929-bbcf-7d8a761ad5c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:16.642074 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4988f57f-ac7d-4929-bbcf-7d8a761ad5c4 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:16.657453 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 508/1007] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:00:16 2022] GET /identity/v3/services?type=image => generated 122 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:17.122899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d293dd2-2142-4a8d-8f4e-3ab5478a7f01 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:17.123209 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d293dd2-2142-4a8d-8f4e-3ab5478a7f01 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:17.123566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d293dd2-2142-4a8d-8f4e-3ab5478a7f01 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:17.125072 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 500/1008] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 15:00:17.133383 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b05481-9451-4b09-a6c1-98cd2b8802fa None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:17.133921 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b05481-9451-4b09-a6c1-98cd2b8802fa None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:17.134302 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b05481-9451-4b09-a6c1-98cd2b8802fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:17.165358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-a7b05481-9451-4b09-a6c1-98cd2b8802fa None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:17.167300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-a7b05481-9451-4b09-a6c1-98cd2b8802fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:17.299134 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 509/1009] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:17 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:00:17.773064 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d293dd2-2142-4a8d-8f4e-3ab5478a7f01 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:17.815920 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67fdf7b-b9ae-4baf-aa98-f4634e279274 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:17.816631 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67fdf7b-b9ae-4baf-aa98-f4634e279274 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:17.817106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67fdf7b-b9ae-4baf-aa98-f4634e279274 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:17.820488 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 510/1010] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:17.828183 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75dd66-4609-41ee-be13-21da0ba9a466 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:17.828523 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75dd66-4609-41ee-be13-21da0ba9a466 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:17.829997 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75dd66-4609-41ee-be13-21da0ba9a466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:17.849521 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-ab75dd66-4609-41ee-be13-21da0ba9a466 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:17.850840 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-ab75dd66-4609-41ee-be13-21da0ba9a466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:17.896749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:17.935280 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:17.949165 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 511/1011] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:17 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 15:00:17.949514 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d293dd2-2142-4a8d-8f4e-3ab5478a7f01 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:18.009574 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a4129a-35a1-4b24-8afa-6705f7151dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:18.010583 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4129a-35a1-4b24-8afa-6705f7151dc8 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:18.010990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4129a-35a1-4b24-8afa-6705f7151dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:18.011388 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4129a-35a1-4b24-8afa-6705f7151dc8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:18.012320 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a4129a-35a1-4b24-8afa-6705f7151dc8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:18.014660 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a4129a-35a1-4b24-8afa-6705f7151dc8 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:18.030409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-75a4129a-35a1-4b24-8afa-6705f7151dc8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 15:00:18.030409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:18.030409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 15:00:18.030409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 15:00:18.030409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 15:00:18.030409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 15:00:18.030409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 15:00:18.030409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:00:18.030409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 15:00:18.030409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:18.030409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 15:00:18.030409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:18.030409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:18.030409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:18.030409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:18.030409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:18.030409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:18.032185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:18.032185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:18.032185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:18.032185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:18.032185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:18.032185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:18.032185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 15:00:18.032185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 15:00:18.032185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 15:00:18.032185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 15:00:18.032185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:18.032185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:18.032185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:18.032185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:18.032185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:18.032185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:18.034443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:18.034443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:18.034443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:18.034443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:18.034443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:18.034443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:18.034443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:18.034443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:18.034443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:18.034443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:18.034443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 15:00:18.034443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:00:18.034443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 15:00:18.034443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:18.034443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 501/1012] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 15:00:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:18.038814 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75dd66-4609-41ee-be13-21da0ba9a466 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:18.165363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75dd66-4609-41ee-be13-21da0ba9a466 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:18.208282 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec4302-8f15-4a51-8251-fbe9f1851d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:18.209253 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec4302-8f15-4a51-8251-fbe9f1851d75 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:18.209846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec4302-8f15-4a51-8251-fbe9f1851d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:18.210409 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec4302-8f15-4a51-8251-fbe9f1851d75 None admin] PATH_INFO: `/v3/services` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:18.211255 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec4302-8f15-4a51-8251-fbe9f1851d75 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:18.211730 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec4302-8f15-4a51-8251-fbe9f1851d75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:18.213437 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec4302-8f15-4a51-8251-fbe9f1851d75 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:18.224253 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 512/1013] 10.222.0.44 () {56 vars in 1112 bytes} [Sun Sep 11 15:00:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:18.226286 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a4129a-35a1-4b24-8afa-6705f7151dc8 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:18.370141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec4302-8f15-4a51-8251-fbe9f1851d75 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:18.402193 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a4129a-35a1-4b24-8afa-6705f7151dc8 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:18.428009 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:18.457220 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807b6bd-3b59-4ce5-ab1c-31306a7a52b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:18.458136 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807b6bd-3b59-4ce5-ab1c-31306a7a52b9 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:18.458573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807b6bd-3b59-4ce5-ab1c-31306a7a52b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:18.458963 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807b6bd-3b59-4ce5-ab1c-31306a7a52b9 None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:18.459747 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807b6bd-3b59-4ce5-ab1c-31306a7a52b9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:18.460167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807b6bd-3b59-4ce5-ab1c-31306a7a52b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:18.462024 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807b6bd-3b59-4ce5-ab1c-31306a7a52b9 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:18.473205 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 502/1014] 10.222.0.44 () {56 vars in 1112 bytes} [Sun Sep 11 15:00:18 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:18.478268 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807b6bd-3b59-4ce5-ab1c-31306a7a52b9 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:18.522345 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:18.533288 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec4302-8f15-4a51-8251-fbe9f1851d75 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:18.603130 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eba986-f260-4908-b644-bbabecac5e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:18.604732 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eba986-f260-4908-b644-bbabecac5e7b None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:18.605202 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eba986-f260-4908-b644-bbabecac5e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:18.605629 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eba986-f260-4908-b644-bbabecac5e7b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:18.608070 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eba986-f260-4908-b644-bbabecac5e7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:18.610650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eba986-f260-4908-b644-bbabecac5e7b None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:18.617198 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 513/1015] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:00:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:18.621512 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eba986-f260-4908-b644-bbabecac5e7b None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:18.645113 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807b6bd-3b59-4ce5-ab1c-31306a7a52b9 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:18.704536 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea94875-fe71-4fe9-968d-686441a910ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:18.705556 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea94875-fe71-4fe9-968d-686441a910ff None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:18.705949 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea94875-fe71-4fe9-968d-686441a910ff None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:18.706366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea94875-fe71-4fe9-968d-686441a910ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:18.707499 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea94875-fe71-4fe9-968d-686441a910ff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:18.709785 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea94875-fe71-4fe9-968d-686441a910ff None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:18.741186 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 503/1016] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:00:18 2022] POST /identity/v3/endpoints => generated 334 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:00:18.790082 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eba986-f260-4908-b644-bbabecac5e7b None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:18.793621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:18.881204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568e6bc2-c022-4f00-bbe4-8c6fce3c9ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:18.882759 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e6bc2-c022-4f00-bbe4-8c6fce3c9ff0 None admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:18.883487 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e6bc2-c022-4f00-bbe4-8c6fce3c9ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:18.884175 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568e6bc2-c022-4f00-bbe4-8c6fce3c9ff0 None admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:18.885988 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568e6bc2-c022-4f00-bbe4-8c6fce3c9ff0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:18.889186 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568e6bc2-c022-4f00-bbe4-8c6fce3c9ff0 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:18.944694 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 514/1017] 10.222.0.44 () {60 vars in 1141 bytes} [Sun Sep 11 15:00:18 2022] POST /identity/v3/projects => generated 311 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:00:18.996192 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f4b43-4250-4782-b062-99c77f37e8c7 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:18.996726 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f4b43-4250-4782-b062-99c77f37e8c7 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:18.997120 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f4b43-4250-4782-b062-99c77f37e8c7 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:18.999943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 504/1018] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:19.009088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d3b2a-416c-43c7-80a5-60aaf82b1788 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:19.009654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d3b2a-416c-43c7-80a5-60aaf82b1788 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:19.010212 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d3b2a-416c-43c7-80a5-60aaf82b1788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:19.039753 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-9b2d3b2a-416c-43c7-80a5-60aaf82b1788 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:19.041022 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-9b2d3b2a-416c-43c7-80a5-60aaf82b1788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:19.146948 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 515/1019] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:19 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:00:19.631497 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f4b43-4250-4782-b062-99c77f37e8c7 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:19.790533 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:19.832810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f4b43-4250-4782-b062-99c77f37e8c7 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:19.902612 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a1b7b-e964-43bb-b149-0ec6c51d2fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:19.908614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a1b7b-e964-43bb-b149-0ec6c51d2fba None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:19.908614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a1b7b-e964-43bb-b149-0ec6c51d2fba None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:19.908614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a1b7b-e964-43bb-b149-0ec6c51d2fba None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:19.908614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a1b7b-e964-43bb-b149-0ec6c51d2fba None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:19.908614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a1b7b-e964-43bb-b149-0ec6c51d2fba None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:19.941266 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 505/1020] 10.222.0.44 () {60 vars in 1141 bytes} [Sun Sep 11 15:00:19 2022] POST /identity/v3/services => generated 242 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:00:21.342418 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4449d-9e8e-4ec2-9f4c-e4498f917548 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:21.342968 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4449d-9e8e-4ec2-9f4c-e4498f917548 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:21.342968 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4449d-9e8e-4ec2-9f4c-e4498f917548 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:21.344538 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 516/1021] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:21.350407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e65128-5358-482c-a67e-0179d64febc8 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:21.350877 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e65128-5358-482c-a67e-0179d64febc8 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:21.350957 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e65128-5358-482c-a67e-0179d64febc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:21.370349 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-35e65128-5358-482c-a67e-0179d64febc8 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:21.371555 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-35e65128-5358-482c-a67e-0179d64febc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:21.448672 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 506/1022] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:00:21.714724 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4449d-9e8e-4ec2-9f4c-e4498f917548 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:21.815915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:21.853493 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4449d-9e8e-4ec2-9f4c-e4498f917548 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:21.916069 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2433b81-0827-495b-999e-5eb150ab6845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:21.917195 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2433b81-0827-495b-999e-5eb150ab6845 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:21.917704 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2433b81-0827-495b-999e-5eb150ab6845 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:21.918170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2433b81-0827-495b-999e-5eb150ab6845 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:21.937392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2433b81-0827-495b-999e-5eb150ab6845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 15:00:21.947064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-a2433b81-0827-495b-999e-5eb150ab6845 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 15:00:21.947064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:21.947064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:21.947064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:21.947064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:21.947064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:21.947064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:21.947064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:21.947064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:21.947064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:21.947064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:21.947064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:21.947064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 15:00:21.947064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 15:00:21.947064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 15:00:21.947064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 15:00:21.948146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 15:00:21.948146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 15:00:21.948146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 15:00:21.948146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 15:00:21.948146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:21.948146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:21.948146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:21.948146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:21.948146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:21.948146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:21.948146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:21.948146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:21.948146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:21.948146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:21.948146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:21.948146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:21.948974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:21.948974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:21.948974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:21.948974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:21.948974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 15:00:21.948974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 15:00:21.948974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 15:00:21.948974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 15:00:21.948974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 15:00:21.948974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 15:00:21.948974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 15:00:21.948974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:21.950500 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 517/1023] 10.222.0.44 () {56 vars in 1120 bytes} [Sun Sep 11 15:00:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:21.956686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5feef1-9b58-4b0e-b486-742a32226466 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:21.957204 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5feef1-9b58-4b0e-b486-742a32226466 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:21.957688 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5feef1-9b58-4b0e-b486-742a32226466 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:21.958216 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2433b81-0827-495b-999e-5eb150ab6845 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:21.960425 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 507/1024] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:21.971066 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bcd33-419b-45e9-9295-24d249178b32 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:21.971440 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bcd33-419b-45e9-9295-24d249178b32 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:21.971925 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bcd33-419b-45e9-9295-24d249178b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:22.001827 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-140bcd33-419b-45e9-9295-24d249178b32 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:22.003848 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-140bcd33-419b-45e9-9295-24d249178b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:22.072722 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:22.082034 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2433b81-0827-495b-999e-5eb150ab6845 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:22.083366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 508/1025] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 15:00:22.122748 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6bfd42-bafd-461f-b007-678d75ba36a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:22.123556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6bfd42-bafd-461f-b007-678d75ba36a1 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:22.123996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6bfd42-bafd-461f-b007-678d75ba36a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:22.124392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6bfd42-bafd-461f-b007-678d75ba36a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:22.125274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6bfd42-bafd-461f-b007-678d75ba36a1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:22.125718 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6bfd42-bafd-461f-b007-678d75ba36a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:22.127440 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6bfd42-bafd-461f-b007-678d75ba36a1 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:22.142417 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 518/1026] 10.222.0.44 () {56 vars in 1115 bytes} [Sun Sep 11 15:00:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:22.143706 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bcd33-419b-45e9-9295-24d249178b32 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:22.262492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bcd33-419b-45e9-9295-24d249178b32 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:22.306448 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d6bb7-1f6b-4f56-b13d-413f85b16607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:22.307285 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d6bb7-1f6b-4f56-b13d-413f85b16607 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:22.307682 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d6bb7-1f6b-4f56-b13d-413f85b16607 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:22.308094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d6bb7-1f6b-4f56-b13d-413f85b16607 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:22.320619 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d6bb7-1f6b-4f56-b13d-413f85b16607 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:22.322357 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d6bb7-1f6b-4f56-b13d-413f85b16607 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:22.339667 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-061d6bb7-1f6b-4f56-b13d-413f85b16607 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:00:22.339667 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:22.339667 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:22.339667 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:22.339667 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:22.339667 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:22.339667 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:22.339667 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:22.339667 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:22.339667 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:22.339667 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:22.339667 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:22.339667 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 15:00:22.339667 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 15:00:22.339667 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 15:00:22.339667 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 15:00:22.341550 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:22.341550 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:22.341550 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:22.341550 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:22.341550 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:22.341550 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:22.341550 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:22.341550 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:22.341550 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:22.341550 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:22.341550 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:22.341550 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:22.341550 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:22.341550 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:22.341550 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:22.341550 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:22.341550 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 15:00:22.343269 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 15:00:22.343269 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:00:22.343269 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:22.345711 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 509/1027] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:00:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 15:00:22.349060 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6bfd42-bafd-461f-b007-678d75ba36a1 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:22.352628 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d6bb7-1f6b-4f56-b13d-413f85b16607 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:22.354894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:22.433908 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:22.504232 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d6bb7-1f6b-4f56-b13d-413f85b16607 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:22.508091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6bfd42-bafd-461f-b007-678d75ba36a1 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:22.548722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf05d3-4d30-457a-9262-3e04ad7ed458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:22.549494 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf05d3-4d30-457a-9262-3e04ad7ed458 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:22.549845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e770718-7b85-406d-9e91-63885103b529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:22.550024 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf05d3-4d30-457a-9262-3e04ad7ed458 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:22.550351 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf05d3-4d30-457a-9262-3e04ad7ed458 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:22.550503 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e770718-7b85-406d-9e91-63885103b529 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:22.550782 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e770718-7b85-406d-9e91-63885103b529 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:22.551035 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e770718-7b85-406d-9e91-63885103b529 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:22.551417 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf05d3-4d30-457a-9262-3e04ad7ed458 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:22.551873 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf05d3-4d30-457a-9262-3e04ad7ed458 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:22.552320 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e770718-7b85-406d-9e91-63885103b529 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:22.553614 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf05d3-4d30-457a-9262-3e04ad7ed458 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:22.553892 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e770718-7b85-406d-9e91-63885103b529 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:22.555881 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 510/1028] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:00:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:22.562105 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e770718-7b85-406d-9e91-63885103b529 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:22.577309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 519/1029] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 15:00:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:22.578596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf05d3-4d30-457a-9262-3e04ad7ed458 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:22.667003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:22.698240 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e770718-7b85-406d-9e91-63885103b529 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:22.717858 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf05d3-4d30-457a-9262-3e04ad7ed458 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:22.720531 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:22.749059 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47da46-8878-44e0-8303-c6e9d3432c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:22.750012 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47da46-8878-44e0-8303-c6e9d3432c4c None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:22.750413 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47da46-8878-44e0-8303-c6e9d3432c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:22.751086 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47da46-8878-44e0-8303-c6e9d3432c4c None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:22.751854 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e47da46-8878-44e0-8303-c6e9d3432c4c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:22.753706 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e47da46-8878-44e0-8303-c6e9d3432c4c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:22.763686 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b00ff-def2-4fee-a6fa-e076e12fb45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:22.764335 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b00ff-def2-4fee-a6fa-e076e12fb45d None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:22.764612 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b00ff-def2-4fee-a6fa-e076e12fb45d None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:22.764865 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b00ff-def2-4fee-a6fa-e076e12fb45d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:22.774497 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b00ff-def2-4fee-a6fa-e076e12fb45d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:22.776048 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b00ff-def2-4fee-a6fa-e076e12fb45d None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:22.782644 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 511/1030] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 15:00:22 2022] POST /identity/v3/users => generated 285 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:00:22.789639 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-8a0b00ff-def2-4fee-a6fa-e076e12fb45d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 15:00:22.789639 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:22.789639 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:22.789639 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:22.789639 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:22.789639 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:22.789639 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:22.789639 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:22.789639 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:22.789639 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:22.789639 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:22.789639 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:22.789639 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 15:00:22.789639 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 15:00:22.789639 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 15:00:22.789639 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 15:00:22.799101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:22.799101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:22.799101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 15:00:22.799101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 15:00:22.799101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 15:00:22.799101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 15:00:22.799101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:22.799101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:22.799101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:22.799101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:22.799101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:22.799101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:22.799101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:22.799101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:22.799101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:22.799101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:22.799101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:22.799101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:22.801314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:22.801314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:22.801314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 15:00:22.801314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 15:00:22.801314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 15:00:22.801314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 15:00:22.801314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 15:00:22.801314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 15:00:22.801314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 15:00:22.801314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:22.801314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 520/1031] 10.222.0.44 () {56 vars in 1093 bytes} [Sun Sep 11 15:00:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 15:00:22.802371 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47da46-8878-44e0-8303-c6e9d3432c4c None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:22.926696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47da46-8878-44e0-8303-c6e9d3432c4c None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:22.969253 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae67513-bf78-46f1-bc91-84e0fa61f318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:22.969898 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae67513-bf78-46f1-bc91-84e0fa61f318 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:22.970577 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae67513-bf78-46f1-bc91-84e0fa61f318 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:22.970830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae67513-bf78-46f1-bc91-84e0fa61f318 None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:22.971587 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae67513-bf78-46f1-bc91-84e0fa61f318 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:22.971886 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae67513-bf78-46f1-bc91-84e0fa61f318 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:22.973412 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae67513-bf78-46f1-bc91-84e0fa61f318 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:23.001185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 512/1032] 10.222.0.44 () {56 vars in 1133 bytes} [Sun Sep 11 15:00:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:23.005117 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b00ff-def2-4fee-a6fa-e076e12fb45d None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:23.128268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b00ff-def2-4fee-a6fa-e076e12fb45d None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:23.156387 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:23.169610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366050f3-7f2d-4eb2-9ed8-a2b561f6240c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:23.170269 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366050f3-7f2d-4eb2-9ed8-a2b561f6240c None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:23.170549 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366050f3-7f2d-4eb2-9ed8-a2b561f6240c None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:23.170814 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366050f3-7f2d-4eb2-9ed8-a2b561f6240c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:23.263334 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366050f3-7f2d-4eb2-9ed8-a2b561f6240c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:23.264826 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366050f3-7f2d-4eb2-9ed8-a2b561f6240c None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:23.377343 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cde545-4e1c-4ccf-85d4-1014614718d6 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:23.378191 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cde545-4e1c-4ccf-85d4-1014614718d6 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:23.378191 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cde545-4e1c-4ccf-85d4-1014614718d6 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:23.379590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 513/1033] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 15:00:23.385285 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c5a89-8d48-479d-a729-dfcc16367d42 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:23.385682 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c5a89-8d48-479d-a729-dfcc16367d42 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:23.385890 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c5a89-8d48-479d-a729-dfcc16367d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:23.405624 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-6c0c5a89-8d48-479d-a729-dfcc16367d42 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:23.406839 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-6c0c5a89-8d48-479d-a729-dfcc16367d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:23.436684 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:23.447741 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 521/1034] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 15:00:23 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 15:00:23.455154 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366050f3-7f2d-4eb2-9ed8-a2b561f6240c None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:23.488510 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:23.518973 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 514/1035] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 15:00:23.607398 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366050f3-7f2d-4eb2-9ed8-a2b561f6240c None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:23.667175 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7617092-78de-4599-bc9f-f09bec8bcfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:23.668172 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7617092-78de-4599-bc9f-f09bec8bcfc5 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:23.668628 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7617092-78de-4599-bc9f-f09bec8bcfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:23.669088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7617092-78de-4599-bc9f-f09bec8bcfc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:23.687973 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7617092-78de-4599-bc9f-f09bec8bcfc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:23.690738 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7617092-78de-4599-bc9f-f09bec8bcfc5 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:23.717653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-a7617092-78de-4599-bc9f-f09bec8bcfc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:00:23.717653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:23.717653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:23.717653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:23.717653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:23.717653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:23.717653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:23.717653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:23.717653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:23.717653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:23.717653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:23.717653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:23.717653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 15:00:23.717653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 15:00:23.717653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 15:00:23.717653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 15:00:23.719095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:23.719095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:23.719095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:23.719095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:23.719095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:23.719095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:23.719095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:23.719095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:23.719095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:23.719095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:23.719095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:23.719095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:23.719095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:23.719095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:23.719095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:23.719095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:23.719095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 15:00:23.725807 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 15:00:23.725807 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:00:23.725807 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:23.725807 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 522/1036] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:00:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 15:00:23.726434 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c5a89-8d48-479d-a729-dfcc16367d42 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:23.788938 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7617092-78de-4599-bc9f-f09bec8bcfc5 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:23.856388 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c5a89-8d48-479d-a729-dfcc16367d42 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:23.887134 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:23.896898 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253d30a-0c79-49e1-978c-08e6d35d38d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:23.897597 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253d30a-0c79-49e1-978c-08e6d35d38d9 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:23.897849 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253d30a-0c79-49e1-978c-08e6d35d38d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:23.898125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253d30a-0c79-49e1-978c-08e6d35d38d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:23.898767 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253d30a-0c79-49e1-978c-08e6d35d38d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:23.899084 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253d30a-0c79-49e1-978c-08e6d35d38d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:23.900520 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253d30a-0c79-49e1-978c-08e6d35d38d9 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:23.919911 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 515/1037] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 15:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:23.921436 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253d30a-0c79-49e1-978c-08e6d35d38d9 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:23.924263 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7617092-78de-4599-bc9f-f09bec8bcfc5 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:23.947021 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:23.963012 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19caca4-55ec-4ff0-a083-4d575ffe6cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:23.964088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19caca4-55ec-4ff0-a083-4d575ffe6cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:23.964428 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19caca4-55ec-4ff0-a083-4d575ffe6cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:23.964782 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19caca4-55ec-4ff0-a083-4d575ffe6cc0 None admin] PATH_INFO: `/v3/services/image` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:23.965494 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19caca4-55ec-4ff0-a083-4d575ffe6cc0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:23.966888 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19caca4-55ec-4ff0-a083-4d575ffe6cc0 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:23.977583 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-f19caca4-55ec-4ff0-a083-4d575ffe6cc0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 15:00:23.977583 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:23.977583 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 15:00:23.977583 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 15:00:23.977583 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 15:00:23.977583 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 15:00:23.977583 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 15:00:23.977583 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:00:23.977583 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 15:00:23.977583 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:23.977583 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 15:00:23.977583 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:23.977583 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:23.977583 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:23.977583 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:23.977583 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:23.977583 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:23.979332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:23.979332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:23.979332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:23.979332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:23.979332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:23.979332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:23.979332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 15:00:23.979332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 15:00:23.979332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 15:00:23.979332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 15:00:23.979332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:23.979332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:23.979332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:23.979332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:23.979332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:23.979332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:23.980236 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:23.980236 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:23.980236 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:23.980236 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:23.980236 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:23.980236 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:23.980236 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:23.980236 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:23.980236 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:23.980236 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:23.980236 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 15:00:23.980236 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:00:23.980236 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 15:00:23.980236 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:23.980236 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 523/1038] 10.222.0.44 () {56 vars in 1105 bytes} [Sun Sep 11 15:00:23 2022] GET /identity/v3/services/image => generated 86 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 15:00:23.984809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19caca4-55ec-4ff0-a083-4d575ffe6cc0 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:24.040945 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253d30a-0c79-49e1-978c-08e6d35d38d9 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:24.078722 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef149934-9b97-4a06-bf0d-4dae7e8c94c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:24.079362 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef149934-9b97-4a06-bf0d-4dae7e8c94c1 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:24.079633 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef149934-9b97-4a06-bf0d-4dae7e8c94c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:24.079899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef149934-9b97-4a06-bf0d-4dae7e8c94c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:24.091470 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef149934-9b97-4a06-bf0d-4dae7e8c94c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:24.092959 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef149934-9b97-4a06-bf0d-4dae7e8c94c1 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:24.106188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-ef149934-9b97-4a06-bf0d-4dae7e8c94c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 15:00:24.106188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:24.106188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:24.106188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:24.106188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:24.106188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:24.106188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:24.106188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:24.106188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:24.106188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:24.106188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:24.106188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:24.106188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 15:00:24.106188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 15:00:24.106188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 15:00:24.107201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 15:00:24.107201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:24.107201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:24.107201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:24.107201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:24.107201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:24.107201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:24.107201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:24.107201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:24.107201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:24.107201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:24.107201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:24.107201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:24.107201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:24.107201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:24.107201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:24.107201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:24.108195 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 15:00:24.108195 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 15:00:24.108195 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 15:00:24.108195 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 15:00:24.108195 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 15:00:24.108195 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 15:00:24.108195 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 15:00:24.108195 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:24.108195 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 516/1039] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 15:00:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 15:00:24.113236 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef149934-9b97-4a06-bf0d-4dae7e8c94c1 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:24.121018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19caca4-55ec-4ff0-a083-4d575ffe6cc0 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:24.160036 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5308426-d1b4-403c-abfe-07fc6f6818be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:24.160653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5308426-d1b4-403c-abfe-07fc6f6818be None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:24.161354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5308426-d1b4-403c-abfe-07fc6f6818be None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:24.161354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5308426-d1b4-403c-abfe-07fc6f6818be None admin] PATH_INFO: `/v3/services` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:24.161969 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5308426-d1b4-403c-abfe-07fc6f6818be None admin] RBAC: Adding query filter params (name=image) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:24.162270 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5308426-d1b4-403c-abfe-07fc6f6818be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:24.163669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5308426-d1b4-403c-abfe-07fc6f6818be None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:24.169578 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 524/1040] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:00:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:24.175330 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5308426-d1b4-403c-abfe-07fc6f6818be None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:24.180444 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:24.218022 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:24.249440 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef149934-9b97-4a06-bf0d-4dae7e8c94c1 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:24.290334 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefdcb4-428e-455f-8648-9244213d34f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:24.291693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefdcb4-428e-455f-8648-9244213d34f3 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:24.291693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefdcb4-428e-455f-8648-9244213d34f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:24.291693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefdcb4-428e-455f-8648-9244213d34f3 None admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:24.292379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cefdcb4-428e-455f-8648-9244213d34f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:24.292682 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cefdcb4-428e-455f-8648-9244213d34f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:24.294320 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cefdcb4-428e-455f-8648-9244213d34f3 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:24.317066 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 517/1041] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 15:00:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:24.318454 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefdcb4-428e-455f-8648-9244213d34f3 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:24.328783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5308426-d1b4-403c-abfe-07fc6f6818be None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:24.377316 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80092b54-d853-48ba-a71f-12da4988382f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:24.378058 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80092b54-d853-48ba-a71f-12da4988382f None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:24.378335 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80092b54-d853-48ba-a71f-12da4988382f None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:24.378603 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80092b54-d853-48ba-a71f-12da4988382f None admin] PATH_INFO: `/v3/services` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:24.379539 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80092b54-d853-48ba-a71f-12da4988382f None admin] RBAC: Adding query filter params (type=image) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:24.379867 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80092b54-d853-48ba-a71f-12da4988382f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:24.381302 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80092b54-d853-48ba-a71f-12da4988382f None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:24.391497 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 525/1042] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:00:24 2022] GET /identity/v3/services?type=image => generated 350 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:24.393517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80092b54-d853-48ba-a71f-12da4988382f None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:24.464636 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefdcb4-428e-455f-8648-9244213d34f3 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:24.473802 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:24.505366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb598ac-29a1-43b2-b824-c23b5877afe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:24.505980 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb598ac-29a1-43b2-b824-c23b5877afe3 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:24.506255 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb598ac-29a1-43b2-b824-c23b5877afe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:24.506520 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb598ac-29a1-43b2-b824-c23b5877afe3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:24.507277 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb598ac-29a1-43b2-b824-c23b5877afe3 None admin] RBAC: Adding query filter params (role.id=92fb89d658dc4d4788cef145280a11f7, scope.project.id=70ac515938a24c45ae6302dbac46a1ab, user.id=453297dac64046d19fc1c331d2d85abc) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:24.507578 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb598ac-29a1-43b2-b824-c23b5877afe3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:24.509056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb598ac-29a1-43b2-b824-c23b5877afe3 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:24.515645 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80092b54-d853-48ba-a71f-12da4988382f None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:24.517033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 518/1043] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 15:00:24 2022] GET /identity/v3/role_assignments?user.id=453297dac64046d19fc1c331d2d85abc&scope.project.id=70ac515938a24c45ae6302dbac46a1ab&role.id=92fb89d658dc4d4788cef145280a11f7 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:24.571479 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0aecd1-f56e-404b-94c7-7fa6b4aeb7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:24.572115 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0aecd1-f56e-404b-94c7-7fa6b4aeb7b4 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:24.572462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0aecd1-f56e-404b-94c7-7fa6b4aeb7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:24.572735 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0aecd1-f56e-404b-94c7-7fa6b4aeb7b4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:24.573504 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0aecd1-f56e-404b-94c7-7fa6b4aeb7b4 None admin] RBAC: Adding query filter params (interface=public, service_id=65b0f1a78fe74f3a87967e393b6d9944, region_id=RegionOne) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:24.573817 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0aecd1-f56e-404b-94c7-7fa6b4aeb7b4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:24.575250 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0aecd1-f56e-404b-94c7-7fa6b4aeb7b4 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:24.585707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 526/1044] 10.222.0.44 () {56 vars in 1251 bytes} [Sun Sep 11 15:00:24 2022] GET /identity/v3/endpoints?service_id=65b0f1a78fe74f3a87967e393b6d9944&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:24.589971 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb598ac-29a1-43b2-b824-c23b5877afe3 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:24.594304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:24.729516 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb598ac-29a1-43b2-b824-c23b5877afe3 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:24.774623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6f9ef-448c-4454-85ce-91596e61d91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:24.775248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6f9ef-448c-4454-85ce-91596e61d91b None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:24.775521 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6f9ef-448c-4454-85ce-91596e61d91b None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:24.775823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6f9ef-448c-4454-85ce-91596e61d91b None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:24.776444 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6f9ef-448c-4454-85ce-91596e61d91b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:24.777846 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6f9ef-448c-4454-85ce-91596e61d91b None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:24.785280 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 519/1045] 10.222.0.44 () {56 vars in 1087 bytes} [Sun Sep 11 15:00:24 2022] GET /identity/v3/services => generated 1957 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:00:25.165411 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903466e-7817-4cd8-8af5-956ee553983c None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:25.165690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903466e-7817-4cd8-8af5-956ee553983c None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:25.166062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903466e-7817-4cd8-8af5-956ee553983c None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:25.167606 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 527/1046] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:25.173441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428cb13b-0ffa-4c0d-a67d-d21748d2cc07 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:25.173814 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428cb13b-0ffa-4c0d-a67d-d21748d2cc07 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:25.174260 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428cb13b-0ffa-4c0d-a67d-d21748d2cc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:25.194936 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-428cb13b-0ffa-4c0d-a67d-d21748d2cc07 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:25.196184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-428cb13b-0ffa-4c0d-a67d-d21748d2cc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:25.286283 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:25.297198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 520/1047] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:00:25.568017 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903466e-7817-4cd8-8af5-956ee553983c None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:25.677932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903466e-7817-4cd8-8af5-956ee553983c None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:25.713900 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:25.720616 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef064f-1eac-47cf-aa26-b1fa62c7622d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:25.721310 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef064f-1eac-47cf-aa26-b1fa62c7622d None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:25.721592 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef064f-1eac-47cf-aa26-b1fa62c7622d None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:25.721871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef064f-1eac-47cf-aa26-b1fa62c7622d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:25.733888 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef064f-1eac-47cf-aa26-b1fa62c7622d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 15:00:25.738722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-f9ef064f-1eac-47cf-aa26-b1fa62c7622d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 15:00:25.738722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:25.738722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:25.738722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:25.738722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:25.738722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:25.738722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:25.738722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:25.738722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:25.738722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:25.738722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:25.738722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:25.738722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 15:00:25.738722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 15:00:25.738722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 15:00:25.738722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 15:00:25.739985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 15:00:25.739985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 15:00:25.739985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 15:00:25.739985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 15:00:25.739985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:25.739985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:25.739985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:25.739985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:25.739985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:25.739985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:25.739985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:25.739985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:25.739985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:25.739985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:25.739985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:25.739985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:25.741423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:25.741423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:25.741423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:25.741423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:25.741423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 15:00:25.741423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 15:00:25.741423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 15:00:25.741423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 15:00:25.741423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 15:00:25.741423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 15:00:25.741423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 15:00:25.741423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:25.741423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 528/1048] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 15:00:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:25.748822 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428cb13b-0ffa-4c0d-a67d-d21748d2cc07 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:25.859633 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428cb13b-0ffa-4c0d-a67d-d21748d2cc07 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:25.898096 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e38ba-713b-4759-998c-5245166d97f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:25.898721 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e38ba-713b-4759-998c-5245166d97f9 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:25.898997 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e38ba-713b-4759-998c-5245166d97f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:25.899272 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e38ba-713b-4759-998c-5245166d97f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:25.899948 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80e38ba-713b-4759-998c-5245166d97f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:25.900249 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80e38ba-713b-4759-998c-5245166d97f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:25.901724 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80e38ba-713b-4759-998c-5245166d97f9 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:25.911476 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 521/1049] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 15:00:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:25.917408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef064f-1eac-47cf-aa26-b1fa62c7622d None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:26.031060 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef064f-1eac-47cf-aa26-b1fa62c7622d None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:26.074917 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a639984-e434-4bba-960d-69ee5f523c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:26.075571 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a639984-e434-4bba-960d-69ee5f523c62 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:26.075851 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a639984-e434-4bba-960d-69ee5f523c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:26.076160 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a639984-e434-4bba-960d-69ee5f523c62 None admin] PATH_INFO: `/v3/users/33c8590c6cd849879adcf4f2d5ef9a37` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:26.093174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a639984-e434-4bba-960d-69ee5f523c62 None admin] RBAC: Authorizing `identity:get_user(user_id=33c8590c6cd849879adcf4f2d5ef9a37)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:26.094678 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a639984-e434-4bba-960d-69ee5f523c62 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:26.097075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 529/1050] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 15:00:25 2022] GET /identity/v3/users/33c8590c6cd849879adcf4f2d5ef9a37 => generated 285 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:26.104365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e38ba-713b-4759-998c-5245166d97f9 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:26.202293 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:26.262945 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e38ba-713b-4759-998c-5245166d97f9 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:26.310267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19dd92-1764-4a58-a24a-d95c6c00f7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:26.310899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19dd92-1764-4a58-a24a-d95c6c00f7ce None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:26.311161 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19dd92-1764-4a58-a24a-d95c6c00f7ce None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:26.311437 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19dd92-1764-4a58-a24a-d95c6c00f7ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:26.397271 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19dd92-1764-4a58-a24a-d95c6c00f7ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:26.399016 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19dd92-1764-4a58-a24a-d95c6c00f7ce None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:26.468374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:26.523162 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 522/1051] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 15:00:26 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 15:00:26.524989 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a639984-e434-4bba-960d-69ee5f523c62 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:26.584649 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:26.646547 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a639984-e434-4bba-960d-69ee5f523c62 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:26.686414 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847b168-fc25-4c26-aa1f-d9f6d5cc36da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:26.687241 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847b168-fc25-4c26-aa1f-d9f6d5cc36da None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:26.687653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847b168-fc25-4c26-aa1f-d9f6d5cc36da None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:26.688085 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847b168-fc25-4c26-aa1f-d9f6d5cc36da None admin] PATH_INFO: `/v3/projects/695a092ba6de497f848792cc3b6bef1f` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:26.689452 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847b168-fc25-4c26-aa1f-d9f6d5cc36da None admin] RBAC: Authorizing `identity:get_project(project_id=695a092ba6de497f848792cc3b6bef1f)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:26.691087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847b168-fc25-4c26-aa1f-d9f6d5cc36da None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:26.693261 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 530/1052] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 15:00:26 2022] GET /identity/v3/projects/695a092ba6de497f848792cc3b6bef1f => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:26.699706 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19dd92-1764-4a58-a24a-d95c6c00f7ce None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:26.820296 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19dd92-1764-4a58-a24a-d95c6c00f7ce None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:26.860839 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c9ca4-483c-43f5-8d4c-687665feee35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:26.861671 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c9ca4-483c-43f5-8d4c-687665feee35 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:26.862111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c9ca4-483c-43f5-8d4c-687665feee35 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:26.862520 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c9ca4-483c-43f5-8d4c-687665feee35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:26.863384 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c9ca4-483c-43f5-8d4c-687665feee35 None admin] RBAC: Adding query filter params (role.id=c426875a15834aa5b122e2a62267d15a, scope.project.id=695a092ba6de497f848792cc3b6bef1f, user.id=33c8590c6cd849879adcf4f2d5ef9a37) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:26.863807 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c9ca4-483c-43f5-8d4c-687665feee35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:26.865439 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c9ca4-483c-43f5-8d4c-687665feee35 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:26.877268 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 523/1053] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 15:00:26 2022] GET /identity/v3/role_assignments?user.id=33c8590c6cd849879adcf4f2d5ef9a37&scope.project.id=695a092ba6de497f848792cc3b6bef1f&role.id=c426875a15834aa5b122e2a62267d15a => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:27.015409 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18a134-0aa3-4254-b910-83234f962c92 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:27.015671 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18a134-0aa3-4254-b910-83234f962c92 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:27.016047 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18a134-0aa3-4254-b910-83234f962c92 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:27.017718 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 531/1054] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:27.024307 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177fffd-c793-47b9-aa64-1964130da8f5 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:27.024771 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177fffd-c793-47b9-aa64-1964130da8f5 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:27.025189 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177fffd-c793-47b9-aa64-1964130da8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:27.049463 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-1177fffd-c793-47b9-aa64-1964130da8f5 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:27.050777 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-1177fffd-c793-47b9-aa64-1964130da8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:27.074495 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:27.100069 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56a453-92e5-4221-9e8d-1190a0ee6a64 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:27.100348 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56a453-92e5-4221-9e8d-1190a0ee6a64 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:27.100598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56a453-92e5-4221-9e8d-1190a0ee6a64 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:27.101984 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 532/1055] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 15:00:27.109626 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44c9be-9f71-4fd2-9b39-4897c0a356dc None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:27.109901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44c9be-9f71-4fd2-9b39-4897c0a356dc None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:27.110166 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44c9be-9f71-4fd2-9b39-4897c0a356dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:27.131076 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-0e44c9be-9f71-4fd2-9b39-4897c0a356dc None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:27.132337 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-0e44c9be-9f71-4fd2-9b39-4897c0a356dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:27.141095 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 524/1056] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:00:27.217195 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 533/1057] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 15:00:27.408432 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177fffd-c793-47b9-aa64-1964130da8f5 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:27.488458 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44c9be-9f71-4fd2-9b39-4897c0a356dc None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:27.526623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177fffd-c793-47b9-aa64-1964130da8f5 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:27.572972 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ad952-4d99-4494-bd1c-0085d24018e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:27.573691 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ad952-4d99-4494-bd1c-0085d24018e2 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:27.573969 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ad952-4d99-4494-bd1c-0085d24018e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:27.574242 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ad952-4d99-4494-bd1c-0085d24018e2 None admin] PATH_INFO: `/v3/services/image` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:27.574862 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ad952-4d99-4494-bd1c-0085d24018e2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:27.576316 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ad952-4d99-4494-bd1c-0085d24018e2 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:27.586835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-4b7ad952-4d99-4494-bd1c-0085d24018e2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 15:00:27.586835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:27.586835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 15:00:27.586835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 15:00:27.586835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 15:00:27.586835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 15:00:27.586835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 15:00:27.586835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:00:27.586835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 15:00:27.586835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:27.586835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 15:00:27.586835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:27.586835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:27.586835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:27.586835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:27.586835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:27.586835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:27.588805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:27.588805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:27.588805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:27.588805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:27.588805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:27.588805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:27.588805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 15:00:27.588805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 15:00:27.588805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 15:00:27.588805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 15:00:27.588805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:27.588805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:27.588805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:27.588805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:27.588805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:27.588805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:27.590452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:27.590452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:27.590452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:27.590452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:27.590452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:27.590452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:27.590452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:27.590452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:27.590452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:27.590452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:27.590452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 15:00:27.590452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:00:27.590452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 15:00:27.590452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:27.590452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 525/1058] 10.222.0.44 () {56 vars in 1105 bytes} [Sun Sep 11 15:00:27 2022] GET /identity/v3/services/image => generated 86 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 15:00:27.595557 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ad952-4d99-4494-bd1c-0085d24018e2 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:27.595822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:27.637521 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:27.700252 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44c9be-9f71-4fd2-9b39-4897c0a356dc None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:27.762041 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba6544-e24c-4711-a4e5-415416fad011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:27.763136 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba6544-e24c-4711-a4e5-415416fad011 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:27.763607 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba6544-e24c-4711-a4e5-415416fad011 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:27.764090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba6544-e24c-4711-a4e5-415416fad011 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:27.774502 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ad952-4d99-4494-bd1c-0085d24018e2 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:27.785046 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba6544-e24c-4711-a4e5-415416fad011 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 15:00:27.794936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-2cba6544-e24c-4711-a4e5-415416fad011 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 15:00:27.794936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:27.794936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:27.794936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:27.794936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:27.794936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:27.794936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:27.794936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:27.794936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:27.794936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:27.794936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:27.794936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:27.794936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 15:00:27.794936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 15:00:27.794936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 15:00:27.794936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 15:00:27.796087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 15:00:27.796087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 15:00:27.796087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 15:00:27.796087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 15:00:27.796087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:27.796087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:27.796087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:27.796087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:27.796087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:27.796087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:27.796087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:27.796087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:27.796087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:27.796087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:27.796087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:27.796087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:27.796936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:27.796936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:27.796936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:27.796936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:27.796936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 15:00:27.796936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 15:00:27.796936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 15:00:27.796936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 15:00:27.796936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 15:00:27.796936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 15:00:27.796936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 15:00:27.796936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:27.798654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 534/1059] 10.222.0.44 () {56 vars in 1120 bytes} [Sun Sep 11 15:00:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 15:00:27.818405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba6544-e24c-4711-a4e5-415416fad011 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:27.843128 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b18f7-e230-4e21-9b21-6fecb64b3984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:27.844294 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b18f7-e230-4e21-9b21-6fecb64b3984 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:27.844638 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b18f7-e230-4e21-9b21-6fecb64b3984 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:27.846955 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b18f7-e230-4e21-9b21-6fecb64b3984 None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:27.846955 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b18f7-e230-4e21-9b21-6fecb64b3984 None admin] RBAC: Adding query filter params (name=image) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:27.848414 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b18f7-e230-4e21-9b21-6fecb64b3984 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:27.851004 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b18f7-e230-4e21-9b21-6fecb64b3984 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:27.865203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 526/1060] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:00:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:27.867406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b18f7-e230-4e21-9b21-6fecb64b3984 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:28.011612 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:28.022325 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba6544-e24c-4711-a4e5-415416fad011 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:28.041323 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:28.045328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b18f7-e230-4e21-9b21-6fecb64b3984 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:28.087090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7caf6-9cbb-4096-a20f-738a88d0e176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:28.088199 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7caf6-9cbb-4096-a20f-738a88d0e176 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:28.088666 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7caf6-9cbb-4096-a20f-738a88d0e176 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:28.089160 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7caf6-9cbb-4096-a20f-738a88d0e176 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:28.090368 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7caf6-9cbb-4096-a20f-738a88d0e176 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:28.090755 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7caf6-9cbb-4096-a20f-738a88d0e176 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:28.093365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7caf6-9cbb-4096-a20f-738a88d0e176 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:28.107997 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595348a-ae8a-4544-9c8d-b7dd08622b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:28.109038 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595348a-ae8a-4544-9c8d-b7dd08622b60 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:28.109545 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595348a-ae8a-4544-9c8d-b7dd08622b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:28.110023 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595348a-ae8a-4544-9c8d-b7dd08622b60 None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:28.111185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595348a-ae8a-4544-9c8d-b7dd08622b60 None admin] RBAC: Adding query filter params (type=image) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:28.111715 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595348a-ae8a-4544-9c8d-b7dd08622b60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:28.113200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 535/1061] 10.222.0.44 () {56 vars in 1115 bytes} [Sun Sep 11 15:00:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:28.114229 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595348a-ae8a-4544-9c8d-b7dd08622b60 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:28.117866 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7caf6-9cbb-4096-a20f-738a88d0e176 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:28.131461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 527/1062] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:00:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:28.133622 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595348a-ae8a-4544-9c8d-b7dd08622b60 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:28.258620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7caf6-9cbb-4096-a20f-738a88d0e176 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:28.317246 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec9be0-d98a-415f-97eb-e5bcbe57e430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:28.318234 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec9be0-d98a-415f-97eb-e5bcbe57e430 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:28.318669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec9be0-d98a-415f-97eb-e5bcbe57e430 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:28.319104 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec9be0-d98a-415f-97eb-e5bcbe57e430 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:28.337291 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec9be0-d98a-415f-97eb-e5bcbe57e430 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:28.340437 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec9be0-d98a-415f-97eb-e5bcbe57e430 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:28.340918 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595348a-ae8a-4544-9c8d-b7dd08622b60 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:28.370415 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-98ec9be0-d98a-415f-97eb-e5bcbe57e430 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:00:28.370415 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:28.370415 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:28.370415 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:28.370415 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:28.370415 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:28.370415 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:28.370415 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:28.370415 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:28.370415 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:28.370415 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:28.370415 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:28.370415 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 15:00:28.370415 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 15:00:28.370415 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 15:00:28.370415 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 15:00:28.371524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:28.371524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:28.371524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:28.371524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:28.371524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:28.371524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:28.371524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:28.371524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:28.371524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:28.371524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:28.371524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:28.371524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:28.371524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:28.371524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:28.371524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:28.371524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:28.371524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 15:00:28.373119 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 15:00:28.373119 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:00:28.373119 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:28.375634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 536/1063] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:00:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:28.386181 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec9be0-d98a-415f-97eb-e5bcbe57e430 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:28.389667 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afe092-ea70-4613-af1b-92ed5bebe018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:28.390493 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afe092-ea70-4613-af1b-92ed5bebe018 None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:28.390935 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afe092-ea70-4613-af1b-92ed5bebe018 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:28.391332 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afe092-ea70-4613-af1b-92ed5bebe018 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:28.392194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afe092-ea70-4613-af1b-92ed5bebe018 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:28.393916 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afe092-ea70-4613-af1b-92ed5bebe018 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:28.420544 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 528/1064] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:00:28 2022] POST /identity/v3/endpoints => generated 327 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:00:28.439782 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:28.600349 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec9be0-d98a-415f-97eb-e5bcbe57e430 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:28.670431 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362f8f8-1167-46b8-b737-3eb36f1bfbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:28.671231 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362f8f8-1167-46b8-b737-3eb36f1bfbe7 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:28.671680 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362f8f8-1167-46b8-b737-3eb36f1bfbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:28.672105 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362f8f8-1167-46b8-b737-3eb36f1bfbe7 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:28.673043 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f362f8f8-1167-46b8-b737-3eb36f1bfbe7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:28.673589 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f362f8f8-1167-46b8-b737-3eb36f1bfbe7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:28.675612 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f362f8f8-1167-46b8-b737-3eb36f1bfbe7 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:28.692501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 537/1065] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 15:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:28.699215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afe092-ea70-4613-af1b-92ed5bebe018 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:28.702571 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:28.874735 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afe092-ea70-4613-af1b-92ed5bebe018 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:28.935005 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc88c6f-d118-490e-9209-2e93b3ef11df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:28.936045 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc88c6f-d118-490e-9209-2e93b3ef11df None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:28.936505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc88c6f-d118-490e-9209-2e93b3ef11df None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:28.936972 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc88c6f-d118-490e-9209-2e93b3ef11df None admin] PATH_INFO: `/v3/users/nova` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:28.951295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc88c6f-d118-490e-9209-2e93b3ef11df None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:28.953990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc88c6f-d118-490e-9209-2e93b3ef11df None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:28.976090 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-8cc88c6f-d118-490e-9209-2e93b3ef11df None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 15:00:28.976090 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:28.976090 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:28.976090 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:28.976090 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:28.976090 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:28.976090 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:28.976090 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:28.976090 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:28.976090 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:28.976090 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:28.976090 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:28.976090 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 15:00:28.976090 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 15:00:28.976090 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 15:00:28.976090 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 15:00:28.977505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:28.977505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:28.977505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 15:00:28.977505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 15:00:28.977505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 15:00:28.977505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 15:00:28.977505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:28.977505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:28.977505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:28.977505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:28.977505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:28.977505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:28.977505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:28.977505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:28.977505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:28.977505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:28.977505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:28.977505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:28.978469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:28.978469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:28.978469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 15:00:28.978469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 15:00:28.978469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 15:00:28.978469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 15:00:28.978469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 15:00:28.978469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 15:00:28.978469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 15:00:28.978469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:28.979637 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 529/1066] 10.222.0.44 () {56 vars in 1093 bytes} [Sun Sep 11 15:00:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:28.987519 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362f8f8-1167-46b8-b737-3eb36f1bfbe7 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:29.129953 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:29.161380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362f8f8-1167-46b8-b737-3eb36f1bfbe7 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:29.207736 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa22634-cca4-4724-83e5-c4b36f42cda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:29.208348 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa22634-cca4-4724-83e5-c4b36f42cda4 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:29.208618 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa22634-cca4-4724-83e5-c4b36f42cda4 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:29.208879 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa22634-cca4-4724-83e5-c4b36f42cda4 None admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:29.209657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa22634-cca4-4724-83e5-c4b36f42cda4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:29.209954 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa22634-cca4-4724-83e5-c4b36f42cda4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:29.211406 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa22634-cca4-4724-83e5-c4b36f42cda4 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:29.240605 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 538/1067] 10.222.0.44 () {56 vars in 1133 bytes} [Sun Sep 11 15:00:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:29.247367 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc88c6f-d118-490e-9209-2e93b3ef11df None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:29.324203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:29.365417 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc82517-9299-49ba-b642-f4de45b1b73a None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:29.365685 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc82517-9299-49ba-b642-f4de45b1b73a None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:29.366002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc82517-9299-49ba-b642-f4de45b1b73a None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:29.367302 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 539/1068] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 15:00:29.373080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5466f8-68fb-4508-bbc5-21b51c586261 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:29.373396 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5466f8-68fb-4508-bbc5-21b51c586261 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:29.373671 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5466f8-68fb-4508-bbc5-21b51c586261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:29.385314 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc88c6f-d118-490e-9209-2e93b3ef11df None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:29.392701 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-fd5466f8-68fb-4508-bbc5-21b51c586261 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:29.393935 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-fd5466f8-68fb-4508-bbc5-21b51c586261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:29.445648 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7dd4e-2570-48c9-a332-7d75a7d32f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:29.446690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7dd4e-2570-48c9-a332-7d75a7d32f3d None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:29.447261 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7dd4e-2570-48c9-a332-7d75a7d32f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:29.447634 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7dd4e-2570-48c9-a332-7d75a7d32f3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:29.467812 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e7dd4e-2570-48c9-a332-7d75a7d32f3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:29.470666 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e7dd4e-2570-48c9-a332-7d75a7d32f3d None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:29.499325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-83e7dd4e-2570-48c9-a332-7d75a7d32f3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:00:29.499325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:29.499325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:29.499325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:29.499325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:29.499325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:29.499325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:29.499325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:29.499325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:29.499325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:29.499325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:29.499325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:29.499325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 15:00:29.499325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 15:00:29.499325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 15:00:29.499325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 15:00:29.501460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:29.501460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:29.501460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:29.501460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:29.501460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:29.501460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:29.501460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:29.501460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:29.501460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:29.501460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:29.501460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:29.501460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:29.501460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:29.501460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:29.501460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:29.501460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:29.501460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 15:00:29.503284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 15:00:29.503284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:00:29.503284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:29.503284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 530/1069] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 15:00:29.508001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7dd4e-2570-48c9-a332-7d75a7d32f3d None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:29.533233 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 540/1070] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 15:00:29.713415 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7dd4e-2570-48c9-a332-7d75a7d32f3d None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:29.743072 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:29.784736 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8c2e4-f2a2-46fa-851b-43fc8cba76cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:29.785845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8c2e4-f2a2-46fa-851b-43fc8cba76cd None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:29.786312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8c2e4-f2a2-46fa-851b-43fc8cba76cd None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:29.786793 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8c2e4-f2a2-46fa-851b-43fc8cba76cd None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:29.788001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8c2e4-f2a2-46fa-851b-43fc8cba76cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:29.788539 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8c2e4-f2a2-46fa-851b-43fc8cba76cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:29.791298 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8c2e4-f2a2-46fa-851b-43fc8cba76cd None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:29.821248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 531/1071] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 15:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:29.827120 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5466f8-68fb-4508-bbc5-21b51c586261 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:29.926945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:29.943135 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8c2e4-f2a2-46fa-851b-43fc8cba76cd None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:29.983481 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5466f8-68fb-4508-bbc5-21b51c586261 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:30.047871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d00e668-79a8-408a-bc54-0deb3ed58ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:30.048897 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00e668-79a8-408a-bc54-0deb3ed58ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:30.049375 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00e668-79a8-408a-bc54-0deb3ed58ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:30.049804 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00e668-79a8-408a-bc54-0deb3ed58ec4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:30.062147 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d00e668-79a8-408a-bc54-0deb3ed58ec4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:30.064748 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8c2e4-f2a2-46fa-851b-43fc8cba76cd None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:30.064944 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d00e668-79a8-408a-bc54-0deb3ed58ec4 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:30.086268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-8d00e668-79a8-408a-bc54-0deb3ed58ec4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 15:00:30.086268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:30.086268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:30.086268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:30.086268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:30.086268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:30.086268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:30.086268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:30.086268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:30.086268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:30.086268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:30.086268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:30.086268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 15:00:30.086268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 15:00:30.086268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 15:00:30.087950 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 15:00:30.087950 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:30.087950 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:30.087950 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:30.087950 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:30.087950 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:30.087950 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:30.087950 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:30.087950 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:30.087950 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:30.087950 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:30.087950 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:30.087950 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:30.087950 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:30.087950 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:30.087950 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:30.087950 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:30.098745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 15:00:30.098745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 15:00:30.098745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 15:00:30.098745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 15:00:30.098745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 15:00:30.098745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 15:00:30.098745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 15:00:30.098745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:30.098745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 541/1072] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 15:00:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:30.098745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d00e668-79a8-408a-bc54-0deb3ed58ec4 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:30.117390 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c68508-15e2-48bf-ad09-e3a3c357bc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:30.118143 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c68508-15e2-48bf-ad09-e3a3c357bc3c None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:30.118478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c68508-15e2-48bf-ad09-e3a3c357bc3c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:30.118813 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c68508-15e2-48bf-ad09-e3a3c357bc3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:30.130943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c68508-15e2-48bf-ad09-e3a3c357bc3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 15:00:30.135980 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-c6c68508-15e2-48bf-ad09-e3a3c357bc3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 15:00:30.135980 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:30.135980 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:30.135980 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:30.135980 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:30.135980 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:30.135980 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:30.135980 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:30.135980 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:30.135980 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:30.135980 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:30.135980 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:30.135980 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 15:00:30.135980 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 15:00:30.135980 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 15:00:30.135980 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 15:00:30.137905 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 15:00:30.137905 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 15:00:30.137905 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 15:00:30.137905 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 15:00:30.137905 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:30.137905 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:30.137905 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:30.137905 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:30.137905 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:30.137905 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:30.137905 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:30.137905 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:30.137905 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:30.137905 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:30.137905 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:30.137905 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:30.139649 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:30.139649 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:30.139649 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:30.139649 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:30.139649 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 15:00:30.139649 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 15:00:30.139649 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 15:00:30.139649 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 15:00:30.139649 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 15:00:30.139649 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 15:00:30.139649 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 15:00:30.139649 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:30.142489 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 532/1073] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 15:00:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:30.143501 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c68508-15e2-48bf-ad09-e3a3c357bc3c None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:30.184537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:30.279194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c68508-15e2-48bf-ad09-e3a3c357bc3c None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:30.316795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d00e668-79a8-408a-bc54-0deb3ed58ec4 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:30.324893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4981f3d-b88f-4d29-afe7-5be7e49861da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:30.325774 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4981f3d-b88f-4d29-afe7-5be7e49861da None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:30.326198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4981f3d-b88f-4d29-afe7-5be7e49861da None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:30.326593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4981f3d-b88f-4d29-afe7-5be7e49861da None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:30.327399 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4981f3d-b88f-4d29-afe7-5be7e49861da None admin] RBAC: Adding query filter params (name=admin) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:30.327823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4981f3d-b88f-4d29-afe7-5be7e49861da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:30.329505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4981f3d-b88f-4d29-afe7-5be7e49861da None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:30.345553 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 533/1074] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 15:00:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:30.348964 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4981f3d-b88f-4d29-afe7-5be7e49861da None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:30.370398 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:30.386616 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480b3e5-b879-4093-adcc-05c8bd1506f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:30.387682 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480b3e5-b879-4093-adcc-05c8bd1506f3 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:30.388135 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480b3e5-b879-4093-adcc-05c8bd1506f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:30.388843 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480b3e5-b879-4093-adcc-05c8bd1506f3 None admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:30.389760 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2480b3e5-b879-4093-adcc-05c8bd1506f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:30.390298 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2480b3e5-b879-4093-adcc-05c8bd1506f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:30.393003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2480b3e5-b879-4093-adcc-05c8bd1506f3 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:30.414281 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 542/1075] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 15:00:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:30.424029 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480b3e5-b879-4093-adcc-05c8bd1506f3 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:30.485939 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:30.494998 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4981f3d-b88f-4d29-afe7-5be7e49861da None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:30.540279 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214d51e-0d95-47e8-a87f-9e1ce1291d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:30.541354 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214d51e-0d95-47e8-a87f-9e1ce1291d60 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:30.541797 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214d51e-0d95-47e8-a87f-9e1ce1291d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:30.542219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214d51e-0d95-47e8-a87f-9e1ce1291d60 None admin] PATH_INFO: `/v3/users/33c8590c6cd849879adcf4f2d5ef9a37` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:30.543870 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7214d51e-0d95-47e8-a87f-9e1ce1291d60 None admin] RBAC: Authorizing `identity:get_user(user_id=33c8590c6cd849879adcf4f2d5ef9a37)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:30.545819 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7214d51e-0d95-47e8-a87f-9e1ce1291d60 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:30.552728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 534/1076] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 15:00:30 2022] GET /identity/v3/users/33c8590c6cd849879adcf4f2d5ef9a37 => generated 285 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:30.554158 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214d51e-0d95-47e8-a87f-9e1ce1291d60 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:30.606010 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480b3e5-b879-4093-adcc-05c8bd1506f3 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:30.659888 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a03098-eaef-46fb-a102-b42b51133fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:30.660564 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a03098-eaef-46fb-a102-b42b51133fca None admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:30.660871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a03098-eaef-46fb-a102-b42b51133fca None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:30.661120 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a03098-eaef-46fb-a102-b42b51133fca None admin] PATH_INFO: `/v3/projects/70ac515938a24c45ae6302dbac46a1ab/users/453297dac64046d19fc1c331d2d85abc/roles/92fb89d658dc4d4788cef145280a11f7` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:30.663456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a03098-eaef-46fb-a102-b42b51133fca None admin] RBAC: Authorizing `identity:create_grant(project_id=70ac515938a24c45ae6302dbac46a1ab, user_id=453297dac64046d19fc1c331d2d85abc, role_id=92fb89d658dc4d4788cef145280a11f7)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:30.664909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a03098-eaef-46fb-a102-b42b51133fca None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:30.681198 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 543/1077] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 15:00:30 2022] PUT /identity/v3/projects/70ac515938a24c45ae6302dbac46a1ab/users/453297dac64046d19fc1c331d2d85abc/roles/92fb89d658dc4d4788cef145280a11f7 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:00:30.739172 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7214d51e-0d95-47e8-a87f-9e1ce1291d60 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:30.808611 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeac512-82e4-4f3f-9281-7ea6f615e0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:30.810048 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeac512-82e4-4f3f-9281-7ea6f615e0dc None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:30.810760 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeac512-82e4-4f3f-9281-7ea6f615e0dc None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:30.811537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeac512-82e4-4f3f-9281-7ea6f615e0dc None admin] PATH_INFO: `/v3/projects/695a092ba6de497f848792cc3b6bef1f` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:30.814283 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeac512-82e4-4f3f-9281-7ea6f615e0dc None admin] RBAC: Authorizing `identity:get_project(project_id=695a092ba6de497f848792cc3b6bef1f)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:30.817794 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeac512-82e4-4f3f-9281-7ea6f615e0dc None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:30.821773 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 535/1078] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 15:00:30 2022] GET /identity/v3/projects/695a092ba6de497f848792cc3b6bef1f => generated 311 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:30.830622 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a03098-eaef-46fb-a102-b42b51133fca None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:30.906368 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:31.013494 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a03098-eaef-46fb-a102-b42b51133fca None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:31.080332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21baf91-a848-48cc-a5ae-94d910ca72fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:31.081405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21baf91-a848-48cc-a5ae-94d910ca72fa None admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:31.081898 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21baf91-a848-48cc-a5ae-94d910ca72fa None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:31.082368 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21baf91-a848-48cc-a5ae-94d910ca72fa None admin] PATH_INFO: `/v3/projects/695a092ba6de497f848792cc3b6bef1f/users/33c8590c6cd849879adcf4f2d5ef9a37/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:31.086056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21baf91-a848-48cc-a5ae-94d910ca72fa None admin] RBAC: Authorizing `identity:create_grant(project_id=695a092ba6de497f848792cc3b6bef1f, user_id=33c8590c6cd849879adcf4f2d5ef9a37, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:31.088922 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21baf91-a848-48cc-a5ae-94d910ca72fa None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:31.121175 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 544/1079] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 15:00:30 2022] PUT /identity/v3/projects/695a092ba6de497f848792cc3b6bef1f/users/33c8590c6cd849879adcf4f2d5ef9a37/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:00:31.187542 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bea5b-b8ed-469a-89bd-e3a0f8c4a62c None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:31.188321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bea5b-b8ed-469a-89bd-e3a0f8c4a62c None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:31.188321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bea5b-b8ed-469a-89bd-e3a0f8c4a62c None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:31.189708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 536/1080] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 15:00:31.196710 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96978695-e5f7-410a-ae4e-88d3418ce253 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:31.197328 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96978695-e5f7-410a-ae4e-88d3418ce253 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:31.197833 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96978695-e5f7-410a-ae4e-88d3418ce253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:31.229158 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-96978695-e5f7-410a-ae4e-88d3418ce253 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:31.230430 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-96978695-e5f7-410a-ae4e-88d3418ce253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:31.363835 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 545/1081] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:00:31.840789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3bea5b-b8ed-469a-89bd-e3a0f8c4a62c None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:31.898345 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:31.987771 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3bea5b-b8ed-469a-89bd-e3a0f8c4a62c None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:32.029635 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632fa530-d308-4263-837e-2ef46b0b22c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:32.030268 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fa530-d308-4263-837e-2ef46b0b22c7 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:32.031158 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fa530-d308-4263-837e-2ef46b0b22c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:32.031158 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fa530-d308-4263-837e-2ef46b0b22c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:32.138007 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632fa530-d308-4263-837e-2ef46b0b22c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:32.140572 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632fa530-d308-4263-837e-2ef46b0b22c7 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:32.193883 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:32.329245 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 537/1082] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 15:00:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 15:00:32.336088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96978695-e5f7-410a-ae4e-88d3418ce253 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:32.390205 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:32.508031 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96978695-e5f7-410a-ae4e-88d3418ce253 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:32.560197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ab70e-2c48-4bab-b25d-65060fee6f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:32.561296 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ab70e-2c48-4bab-b25d-65060fee6f4f None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:32.561795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ab70e-2c48-4bab-b25d-65060fee6f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:32.562235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ab70e-2c48-4bab-b25d-65060fee6f4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:32.575040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ab70e-2c48-4bab-b25d-65060fee6f4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:32.576774 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ab70e-2c48-4bab-b25d-65060fee6f4f None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:32.593388 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-679ab70e-2c48-4bab-b25d-65060fee6f4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:00:32.593388 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:32.593388 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:32.593388 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:32.593388 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:32.593388 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:32.593388 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:32.593388 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:32.593388 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:32.593388 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:32.593388 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:32.593388 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:32.593388 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 15:00:32.593388 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 15:00:32.593388 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 15:00:32.593388 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 15:00:32.594472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:32.594472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:32.594472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:32.594472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:32.594472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:32.594472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:32.594472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:32.594472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:32.594472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:32.594472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:32.594472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:32.594472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:32.594472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:32.594472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:32.594472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:32.594472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:32.594472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 15:00:32.595746 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 15:00:32.595746 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:00:32.595746 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:32.596271 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 546/1083] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:00:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 15:00:32.606222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632fa530-d308-4263-837e-2ef46b0b22c7 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:32.742394 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632fa530-d308-4263-837e-2ef46b0b22c7 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:32.781450 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:32.791104 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334143df-205b-4242-a85c-871a8ab8a710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:32.791747 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334143df-205b-4242-a85c-871a8ab8a710 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:32.792051 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334143df-205b-4242-a85c-871a8ab8a710 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:32.792465 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334143df-205b-4242-a85c-871a8ab8a710 None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:32.793430 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334143df-205b-4242-a85c-871a8ab8a710 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:32.793920 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334143df-205b-4242-a85c-871a8ab8a710 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:32.795794 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334143df-205b-4242-a85c-871a8ab8a710 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:32.815830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 538/1084] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 15:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:32.922329 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664222fe-00af-45ac-b63f-8a991506fbee None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:32.922683 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664222fe-00af-45ac-b63f-8a991506fbee None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:32.922874 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664222fe-00af-45ac-b63f-8a991506fbee None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:32.924468 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 547/1085] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 15:00:32.930382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b820d-03cd-4de2-be23-7ae205211990 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:32.930705 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b820d-03cd-4de2-be23-7ae205211990 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:32.930975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b820d-03cd-4de2-be23-7ae205211990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:32.956607 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-958b820d-03cd-4de2-be23-7ae205211990 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:32.957889 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-958b820d-03cd-4de2-be23-7ae205211990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:33.063069 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 539/1086] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:00:33.326051 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664222fe-00af-45ac-b63f-8a991506fbee None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:33.403622 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62612d-a03a-4895-893f-1b54b93608c2 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:33.404224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62612d-a03a-4895-893f-1b54b93608c2 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:33.404571 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62612d-a03a-4895-893f-1b54b93608c2 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:33.407071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 540/1087] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:33.416699 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0399f5-ba70-432e-9895-ee10656a6818 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:33.417263 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0399f5-ba70-432e-9895-ee10656a6818 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:33.417734 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0399f5-ba70-432e-9895-ee10656a6818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:33.441415 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:33.444534 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-db0399f5-ba70-432e-9895-ee10656a6818 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:33.445838 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-db0399f5-ba70-432e-9895-ee10656a6818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:33.457767 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664222fe-00af-45ac-b63f-8a991506fbee None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:33.499619 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7594b76e-b1a5-4ad9-95cc-e8b042115680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:33.500620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594b76e-b1a5-4ad9-95cc-e8b042115680 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:33.501052 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594b76e-b1a5-4ad9-95cc-e8b042115680 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:33.501527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594b76e-b1a5-4ad9-95cc-e8b042115680 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:33.519288 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7594b76e-b1a5-4ad9-95cc-e8b042115680 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 15:00:33.524251 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-7594b76e-b1a5-4ad9-95cc-e8b042115680 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 15:00:33.524251 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:33.524251 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:33.524251 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:33.524251 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:33.524251 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:33.524251 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:33.524251 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:33.524251 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:33.524251 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:33.524251 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:33.524251 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:33.524251 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 15:00:33.524251 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 15:00:33.524251 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 15:00:33.524251 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 15:00:33.525676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 15:00:33.525676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 15:00:33.525676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 15:00:33.525676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 15:00:33.525676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:33.525676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:33.525676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:33.525676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:33.525676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:33.525676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:33.525676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:33.525676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:33.525676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:33.525676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:33.525676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:33.525676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:33.527742 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:33.527742 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:33.527742 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:33.527742 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:33.527742 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 15:00:33.527742 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 15:00:33.527742 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 15:00:33.527742 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 15:00:33.527742 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 15:00:33.527742 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 15:00:33.527742 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 15:00:33.527742 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:33.527742 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 548/1088] 10.222.0.44 () {56 vars in 1120 bytes} [Sun Sep 11 15:00:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:33.531902 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7594b76e-b1a5-4ad9-95cc-e8b042115680 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:33.533709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 541/1089] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 15:00:33.655733 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7594b76e-b1a5-4ad9-95cc-e8b042115680 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:33.698054 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8718ba1-1698-4d61-8261-61ee0cdbe080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:33.698811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8718ba1-1698-4d61-8261-61ee0cdbe080 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:33.699184 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8718ba1-1698-4d61-8261-61ee0cdbe080 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:33.699537 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8718ba1-1698-4d61-8261-61ee0cdbe080 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:33.700336 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8718ba1-1698-4d61-8261-61ee0cdbe080 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:33.700727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8718ba1-1698-4d61-8261-61ee0cdbe080 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:33.702288 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8718ba1-1698-4d61-8261-61ee0cdbe080 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:33.719295 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 549/1090] 10.222.0.44 () {56 vars in 1115 bytes} [Sun Sep 11 15:00:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:33.721692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0399f5-ba70-432e-9895-ee10656a6818 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:33.806610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8718ba1-1698-4d61-8261-61ee0cdbe080 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:33.819193 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:33.821333 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:33.872333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0399f5-ba70-432e-9895-ee10656a6818 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:33.910528 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb9573-baf5-430e-8811-9704fd29e101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:33.911051 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb9573-baf5-430e-8811-9704fd29e101 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:33.911320 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb9573-baf5-430e-8811-9704fd29e101 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:33.911583 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb9573-baf5-430e-8811-9704fd29e101 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:33.923113 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfb9573-baf5-430e-8811-9704fd29e101 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:33.924607 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfb9573-baf5-430e-8811-9704fd29e101 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:33.941851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-cdfb9573-baf5-430e-8811-9704fd29e101 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:00:33.941851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:33.941851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:33.941851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:33.941851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:33.941851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:33.941851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:33.941851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:33.941851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:33.941851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:33.941851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:33.941851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:33.941851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 15:00:33.941851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 15:00:33.941851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 15:00:33.941851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 15:00:33.942913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:33.942913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:33.942913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:33.942913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:33.942913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:33.942913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:33.942913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:33.942913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:33.942913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:33.942913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:33.942913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:33.942913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:33.942913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:33.942913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:33.942913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:33.942913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:33.942913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 15:00:33.943855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 15:00:33.943855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:00:33.943855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:33.943855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 542/1091] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:00:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:33.949334 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb9573-baf5-430e-8811-9704fd29e101 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:33.954825 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8718ba1-1698-4d61-8261-61ee0cdbe080 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:33.992603 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49039794-93c8-4a8a-89fb-ee228eb7d3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:33.993305 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49039794-93c8-4a8a-89fb-ee228eb7d3cf None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:33.993582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49039794-93c8-4a8a-89fb-ee228eb7d3cf None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:33.993849 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49039794-93c8-4a8a-89fb-ee228eb7d3cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:34.008352 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49039794-93c8-4a8a-89fb-ee228eb7d3cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 15:00:34.013465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-49039794-93c8-4a8a-89fb-ee228eb7d3cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 15:00:34.013465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:34.013465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:34.013465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:34.013465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:34.013465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:34.013465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:34.013465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:34.013465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:34.013465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:34.013465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:34.013465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:34.013465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 15:00:34.013465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 15:00:34.013465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 15:00:34.013465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 15:00:34.015544 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 15:00:34.015544 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 15:00:34.015544 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 15:00:34.015544 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 15:00:34.015544 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:34.015544 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:34.015544 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:34.015544 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:34.015544 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:34.015544 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:34.015544 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:34.015544 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:34.015544 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:34.015544 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:34.015544 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:34.015544 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:34.017491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:34.017491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:34.017491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:34.017491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:34.017491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 15:00:34.017491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 15:00:34.017491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 15:00:34.017491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 15:00:34.017491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 15:00:34.017491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 15:00:34.017491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 15:00:34.017491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:34.017491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 550/1092] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 15:00:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:34.022966 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49039794-93c8-4a8a-89fb-ee228eb7d3cf None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:34.059568 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb9573-baf5-430e-8811-9704fd29e101 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:34.062131 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:34.098560 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4259a-3ef3-4a96-9d65-d292456b57ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:34.099181 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4259a-3ef3-4a96-9d65-d292456b57ff None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:34.099451 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4259a-3ef3-4a96-9d65-d292456b57ff None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:34.099713 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4259a-3ef3-4a96-9d65-d292456b57ff None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:34.100358 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e4259a-3ef3-4a96-9d65-d292456b57ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:34.100660 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e4259a-3ef3-4a96-9d65-d292456b57ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:34.100866 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:34.102081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e4259a-3ef3-4a96-9d65-d292456b57ff None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:34.114856 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 543/1093] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 15:00:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:34.121395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4259a-3ef3-4a96-9d65-d292456b57ff None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:34.135096 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49039794-93c8-4a8a-89fb-ee228eb7d3cf None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:34.171601 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456898e9-6bdb-42fc-94c4-92b5cf66234e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:34.172113 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456898e9-6bdb-42fc-94c4-92b5cf66234e None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:34.172386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456898e9-6bdb-42fc-94c4-92b5cf66234e None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:34.172671 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456898e9-6bdb-42fc-94c4-92b5cf66234e None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:34.173374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456898e9-6bdb-42fc-94c4-92b5cf66234e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:34.173657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456898e9-6bdb-42fc-94c4-92b5cf66234e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:34.175064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456898e9-6bdb-42fc-94c4-92b5cf66234e None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:34.184416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 551/1094] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 15:00:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:34.190795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456898e9-6bdb-42fc-94c4-92b5cf66234e None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:34.230950 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4259a-3ef3-4a96-9d65-d292456b57ff None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:34.280898 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad58d3f-206c-4599-aa01-863f40a8d10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:34.281601 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad58d3f-206c-4599-aa01-863f40a8d10e None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:34.281869 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad58d3f-206c-4599-aa01-863f40a8d10e None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:34.282139 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad58d3f-206c-4599-aa01-863f40a8d10e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:34.291118 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad58d3f-206c-4599-aa01-863f40a8d10e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:34.292618 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad58d3f-206c-4599-aa01-863f40a8d10e None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:34.313517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-4ad58d3f-206c-4599-aa01-863f40a8d10e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 15:00:34.313517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:34.313517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:34.313517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:34.313517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:34.313517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:34.313517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:34.313517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:34.313517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:34.313517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:34.313517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:34.313517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:34.313517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 15:00:34.313517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 15:00:34.313517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 15:00:34.313517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 15:00:34.315014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:34.315014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:34.315014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 15:00:34.315014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 15:00:34.315014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 15:00:34.315014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 15:00:34.315014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:34.315014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:34.315014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:34.315014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:34.315014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:34.315014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:34.315014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:34.315014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:34.315014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:34.315014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:34.315014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:34.315014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:34.316537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:34.316537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:34.316537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 15:00:34.316537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 15:00:34.316537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 15:00:34.316537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 15:00:34.316537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 15:00:34.316537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 15:00:34.316537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 15:00:34.316537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:34.321196 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 544/1095] 10.222.0.44 () {56 vars in 1093 bytes} [Sun Sep 11 15:00:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 15:00:34.323445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456898e9-6bdb-42fc-94c4-92b5cf66234e None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:34.325000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad58d3f-206c-4599-aa01-863f40a8d10e None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:34.355624 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:34.362631 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6927813-343f-4d61-ac04-79056ef646ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:34.363398 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6927813-343f-4d61-ac04-79056ef646ff None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:34.363731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6927813-343f-4d61-ac04-79056ef646ff None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:34.364077 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6927813-343f-4d61-ac04-79056ef646ff None admin] PATH_INFO: `/v3/users/33c8590c6cd849879adcf4f2d5ef9a37` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:34.365535 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6927813-343f-4d61-ac04-79056ef646ff None admin] RBAC: Authorizing `identity:get_user(user_id=33c8590c6cd849879adcf4f2d5ef9a37)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:34.367553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6927813-343f-4d61-ac04-79056ef646ff None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:34.367726 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:34.370054 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 552/1096] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 15:00:34 2022] GET /identity/v3/users/33c8590c6cd849879adcf4f2d5ef9a37 => generated 285 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:34.376697 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6927813-343f-4d61-ac04-79056ef646ff None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:34.458480 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad58d3f-206c-4599-aa01-863f40a8d10e None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:34.493794 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6927813-343f-4d61-ac04-79056ef646ff None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:34.504455 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad3780-0ef4-4984-8a0f-a13732bfef95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:34.505290 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad3780-0ef4-4984-8a0f-a13732bfef95 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:34.505664 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad3780-0ef4-4984-8a0f-a13732bfef95 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:34.506081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad3780-0ef4-4984-8a0f-a13732bfef95 None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:34.506949 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad3780-0ef4-4984-8a0f-a13732bfef95 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:34.507345 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad3780-0ef4-4984-8a0f-a13732bfef95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:34.509054 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad3780-0ef4-4984-8a0f-a13732bfef95 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:34.530891 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 545/1097] 10.222.0.44 () {56 vars in 1133 bytes} [Sun Sep 11 15:00:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:34.531235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f62ef0-8c38-4456-8be4-5c5a959a4371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:34.531826 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f62ef0-8c38-4456-8be4-5c5a959a4371 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:34.532095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f62ef0-8c38-4456-8be4-5c5a959a4371 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:34.532359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f62ef0-8c38-4456-8be4-5c5a959a4371 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:34.537739 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad3780-0ef4-4984-8a0f-a13732bfef95 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:34.626661 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:34.638921 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:34.642048 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f62ef0-8c38-4456-8be4-5c5a959a4371 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:34.644542 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f62ef0-8c38-4456-8be4-5c5a959a4371 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:34.650025 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad3780-0ef4-4984-8a0f-a13732bfef95 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:34.688245 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e42c5-7e3a-4f12-a4f0-e05f8e7ddb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:34.688937 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e42c5-7e3a-4f12-a4f0-e05f8e7ddb4c None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:34.689316 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e42c5-7e3a-4f12-a4f0-e05f8e7ddb4c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:34.689654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e42c5-7e3a-4f12-a4f0-e05f8e7ddb4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:34.773837 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 553/1098] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 15:00:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 15:00:34.774972 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e42c5-7e3a-4f12-a4f0-e05f8e7ddb4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:34.776490 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e42c5-7e3a-4f12-a4f0-e05f8e7ddb4c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:34.778881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f62ef0-8c38-4456-8be4-5c5a959a4371 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:34.868051 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:34.890180 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 546/1099] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 15:00:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 15:00:34.896829 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e42c5-7e3a-4f12-a4f0-e05f8e7ddb4c None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:34.902973 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:34.907143 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f62ef0-8c38-4456-8be4-5c5a959a4371 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:34.945579 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7505111f-88fe-4f99-8364-49328b47fa9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:34.946214 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7505111f-88fe-4f99-8364-49328b47fa9b None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:34.946488 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7505111f-88fe-4f99-8364-49328b47fa9b None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:34.946758 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7505111f-88fe-4f99-8364-49328b47fa9b None admin] PATH_INFO: `/v3/projects/695a092ba6de497f848792cc3b6bef1f` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:34.948126 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7505111f-88fe-4f99-8364-49328b47fa9b None admin] RBAC: Authorizing `identity:get_project(project_id=695a092ba6de497f848792cc3b6bef1f)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:34.949681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7505111f-88fe-4f99-8364-49328b47fa9b None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:34.951512 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 554/1100] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 15:00:34 2022] GET /identity/v3/projects/695a092ba6de497f848792cc3b6bef1f => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:34.957905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7505111f-88fe-4f99-8364-49328b47fa9b None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:35.030405 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e42c5-7e3a-4f12-a4f0-e05f8e7ddb4c None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:35.080993 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512c43c-a0a4-40b0-85eb-42a9bbe51a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:35.081838 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512c43c-a0a4-40b0-85eb-42a9bbe51a1d None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:35.082185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512c43c-a0a4-40b0-85eb-42a9bbe51a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:35.082606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512c43c-a0a4-40b0-85eb-42a9bbe51a1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:35.086399 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7505111f-88fe-4f99-8364-49328b47fa9b None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:35.096859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3512c43c-a0a4-40b0-85eb-42a9bbe51a1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:35.098734 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3512c43c-a0a4-40b0-85eb-42a9bbe51a1d None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:35.120919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-3512c43c-a0a4-40b0-85eb-42a9bbe51a1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:00:35.120919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:35.120919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:35.120919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:35.120919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:35.120919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:35.120919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:35.120919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:35.120919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:35.120919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:35.120919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:35.120919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:35.120919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 15:00:35.120919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 15:00:35.120919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 15:00:35.120919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 15:00:35.122994 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:35.122994 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:35.122994 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:35.122994 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:35.122994 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:35.122994 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:35.122994 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:35.122994 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:35.122994 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:35.122994 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:35.122994 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:35.122994 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:35.122994 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:35.122994 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:35.122994 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:35.122994 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:35.122994 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 15:00:35.124671 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 15:00:35.124671 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:00:35.124671 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:35.124671 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 547/1101] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:00:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 15:00:35.131100 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512c43c-a0a4-40b0-85eb-42a9bbe51a1d None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:35.134543 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c844426-ee0c-4baa-9858-aff329622fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:35.135539 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c844426-ee0c-4baa-9858-aff329622fec None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:35.135974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c844426-ee0c-4baa-9858-aff329622fec None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:35.136394 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c844426-ee0c-4baa-9858-aff329622fec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:35.137249 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c844426-ee0c-4baa-9858-aff329622fec None admin] RBAC: Adding query filter params (role.id=c426875a15834aa5b122e2a62267d15a, scope.project.id=695a092ba6de497f848792cc3b6bef1f, user.id=33c8590c6cd849879adcf4f2d5ef9a37) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:35.137784 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c844426-ee0c-4baa-9858-aff329622fec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:35.139458 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c844426-ee0c-4baa-9858-aff329622fec None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:35.148601 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 555/1102] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 15:00:34 2022] GET /identity/v3/role_assignments?user.id=33c8590c6cd849879adcf4f2d5ef9a37&scope.project.id=695a092ba6de497f848792cc3b6bef1f&role.id=c426875a15834aa5b122e2a62267d15a => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:35.190344 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:35.309578 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512c43c-a0a4-40b0-85eb-42a9bbe51a1d None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:35.351807 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca442765-5322-4969-b8c3-42043b305b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:35.352429 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca442765-5322-4969-b8c3-42043b305b3b None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:35.352702 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca442765-5322-4969-b8c3-42043b305b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:35.352969 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca442765-5322-4969-b8c3-42043b305b3b None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:35.353732 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca442765-5322-4969-b8c3-42043b305b3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:35.354034 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca442765-5322-4969-b8c3-42043b305b3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:35.355442 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca442765-5322-4969-b8c3-42043b305b3b None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:35.370859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 548/1103] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 15:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:35.378240 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c844426-ee0c-4baa-9858-aff329622fec None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:35.427127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:35.541827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c844426-ee0c-4baa-9858-aff329622fec None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:35.581057 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4028aa-9d8f-4be5-aed2-ff3df52fad02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:35.581715 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4028aa-9d8f-4be5-aed2-ff3df52fad02 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:35.582053 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4028aa-9d8f-4be5-aed2-ff3df52fad02 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:35.582385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4028aa-9d8f-4be5-aed2-ff3df52fad02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:35.590305 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4028aa-9d8f-4be5-aed2-ff3df52fad02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:35.591837 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4028aa-9d8f-4be5-aed2-ff3df52fad02 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:35.604855 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-ba4028aa-9d8f-4be5-aed2-ff3df52fad02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 15:00:35.604855 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:35.604855 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:35.604855 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:35.604855 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:35.604855 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:35.604855 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:35.604855 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:35.604855 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:35.604855 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:35.604855 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:35.604855 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:35.604855 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 15:00:35.604855 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 15:00:35.604855 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 15:00:35.606841 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 15:00:35.606841 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:35.606841 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:35.606841 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:35.606841 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:35.606841 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:35.606841 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:35.606841 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:35.606841 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:35.606841 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:35.606841 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:35.606841 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:35.606841 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:35.606841 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:35.606841 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:35.606841 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:35.606841 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:35.609066 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 15:00:35.609066 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 15:00:35.609066 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 15:00:35.609066 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 15:00:35.609066 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 15:00:35.609066 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 15:00:35.609066 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 15:00:35.609066 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:00:35.609066 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 556/1104] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 15:00:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:35.613814 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca442765-5322-4969-b8c3-42043b305b3b None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:35.721959 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:35.734062 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca442765-5322-4969-b8c3-42043b305b3b None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:35.799855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc4472-2f71-47d6-a1a1-2163967add77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:35.800875 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc4472-2f71-47d6-a1a1-2163967add77 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:35.801391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc4472-2f71-47d6-a1a1-2163967add77 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:35.801869 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc4472-2f71-47d6-a1a1-2163967add77 None admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:35.803146 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afc4472-2f71-47d6-a1a1-2163967add77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:35.803641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afc4472-2f71-47d6-a1a1-2163967add77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:35.806425 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afc4472-2f71-47d6-a1a1-2163967add77 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:35.829972 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 549/1105] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 15:00:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:35.836630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4028aa-9d8f-4be5-aed2-ff3df52fad02 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:35.948574 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:35.958886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4028aa-9d8f-4be5-aed2-ff3df52fad02 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:35.997575 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee88edf-12ae-4ad7-a404-65f9ee994317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:35.998187 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee88edf-12ae-4ad7-a404-65f9ee994317 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:35.998456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee88edf-12ae-4ad7-a404-65f9ee994317 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:35.998727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee88edf-12ae-4ad7-a404-65f9ee994317 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:35.999462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee88edf-12ae-4ad7-a404-65f9ee994317 None admin] RBAC: Adding query filter params (role.id=92fb89d658dc4d4788cef145280a11f7, scope.project.id=70ac515938a24c45ae6302dbac46a1ab, user.id=453297dac64046d19fc1c331d2d85abc) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:35.999767 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee88edf-12ae-4ad7-a404-65f9ee994317 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:36.001265 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee88edf-12ae-4ad7-a404-65f9ee994317 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:36.008114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 557/1106] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 15:00:35 2022] GET /identity/v3/role_assignments?user.id=453297dac64046d19fc1c331d2d85abc&scope.project.id=70ac515938a24c45ae6302dbac46a1ab&role.id=92fb89d658dc4d4788cef145280a11f7 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:37.434666 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726208d-cbc1-4948-9448-7113b445f53d None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:37.435239 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726208d-cbc1-4948-9448-7113b445f53d None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:37.435239 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726208d-cbc1-4948-9448-7113b445f53d None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:37.436620 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 550/1107] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:37.444345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb9417-14b1-401b-b12e-6edc69399ad9 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:37.444977 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb9417-14b1-401b-b12e-6edc69399ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:37.445365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb9417-14b1-401b-b12e-6edc69399ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:37.475537 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-f8bb9417-14b1-401b-b12e-6edc69399ad9 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:37.477717 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-f8bb9417-14b1-401b-b12e-6edc69399ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:37.604410 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 558/1108] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:00:37.894913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726208d-cbc1-4948-9448-7113b445f53d None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:38.012176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726208d-cbc1-4948-9448-7113b445f53d None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:38.049359 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce9426d-3678-48db-8daf-a5bb856a9f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:38.050043 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9426d-3678-48db-8daf-a5bb856a9f6b None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:38.050364 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9426d-3678-48db-8daf-a5bb856a9f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:38.050693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9426d-3678-48db-8daf-a5bb856a9f6b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:38.051935 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce9426d-3678-48db-8daf-a5bb856a9f6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:38.053484 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce9426d-3678-48db-8daf-a5bb856a9f6b None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:38.055140 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 551/1109] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:00:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:38.061291 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb9417-14b1-401b-b12e-6edc69399ad9 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:38.149492 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:38.165481 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb9417-14b1-401b-b12e-6edc69399ad9 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:38.207833 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7179e14-1837-4d9d-93c5-495b866fcbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:38.208848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7179e14-1837-4d9d-93c5-495b866fcbc7 None admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:38.209359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7179e14-1837-4d9d-93c5-495b866fcbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:38.209826 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7179e14-1837-4d9d-93c5-495b866fcbc7 None admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:38.210991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7179e14-1837-4d9d-93c5-495b866fcbc7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:38.213719 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7179e14-1837-4d9d-93c5-495b866fcbc7 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:38.247041 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 559/1110] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 15:00:38 2022] POST /identity/v3/users => generated 286 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:00:40.483672 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a6eb42-a8f0-43a4-b1ec-91050f5f03c5 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:40.484243 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a6eb42-a8f0-43a4-b1ec-91050f5f03c5 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:40.484243 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a6eb42-a8f0-43a4-b1ec-91050f5f03c5 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:40.485702 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 552/1111] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:40.491604 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee629c-8b19-4041-b2e7-fb233a411209 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:40.492156 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee629c-8b19-4041-b2e7-fb233a411209 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:40.492156 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee629c-8b19-4041-b2e7-fb233a411209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:40.511219 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-76ee629c-8b19-4041-b2e7-fb233a411209 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:40.512433 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-76ee629c-8b19-4041-b2e7-fb233a411209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:40.593739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 560/1112] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:00:40.868442 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a6eb42-a8f0-43a4-b1ec-91050f5f03c5 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:40.896183 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:41.001495 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a6eb42-a8f0-43a4-b1ec-91050f5f03c5 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:41.046557 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beee995-8905-4f9e-9895-8a85cb311f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:41.047263 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beee995-8905-4f9e-9895-8a85cb311f25 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:41.047583 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beee995-8905-4f9e-9895-8a85cb311f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:41.047897 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beee995-8905-4f9e-9895-8a85cb311f25 None admin] PATH_INFO: `/v3/roles/b78105372b434d86872dd04e63dce750` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:41.049384 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beee995-8905-4f9e-9895-8a85cb311f25 None admin] RBAC: Authorizing `identity:get_role(role_id=b78105372b434d86872dd04e63dce750)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:41.050811 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beee995-8905-4f9e-9895-8a85cb311f25 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:41.052452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 553/1113] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 15:00:40 2022] GET /identity/v3/roles/b78105372b434d86872dd04e63dce750 => generated 223 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:41.059204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee629c-8b19-4041-b2e7-fb233a411209 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:41.133250 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:41.166085 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee629c-8b19-4041-b2e7-fb233a411209 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:41.202866 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036836e-3ae6-474d-b9be-3b0b2d6113d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:41.203577 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036836e-3ae6-474d-b9be-3b0b2d6113d1 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:41.203894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036836e-3ae6-474d-b9be-3b0b2d6113d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:41.204228 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036836e-3ae6-474d-b9be-3b0b2d6113d1 None admin] PATH_INFO: `/v3/users/89a74343be7744b7918e45599ee79b71` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:41.219124 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6036836e-3ae6-474d-b9be-3b0b2d6113d1 None admin] RBAC: Authorizing `identity:get_user(user_id=89a74343be7744b7918e45599ee79b71)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:41.221017 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6036836e-3ae6-474d-b9be-3b0b2d6113d1 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:41.222772 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 561/1114] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 15:00:41 2022] GET /identity/v3/users/89a74343be7744b7918e45599ee79b71 => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:41.228975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beee995-8905-4f9e-9895-8a85cb311f25 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:41.372578 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:41.392885 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beee995-8905-4f9e-9895-8a85cb311f25 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:41.449521 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fbfdc5-76ef-43b9-bc38-3c001e3aa2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:41.450146 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbfdc5-76ef-43b9-bc38-3c001e3aa2d0 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:41.450403 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbfdc5-76ef-43b9-bc38-3c001e3aa2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:41.450662 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbfdc5-76ef-43b9-bc38-3c001e3aa2d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:41.525522 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fbfdc5-76ef-43b9-bc38-3c001e3aa2d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:41.527033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fbfdc5-76ef-43b9-bc38-3c001e3aa2d0 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:41.624368 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:41.637930 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 554/1115] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 15:00:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 15:00:41.644435 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036836e-3ae6-474d-b9be-3b0b2d6113d1 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:41.751639 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036836e-3ae6-474d-b9be-3b0b2d6113d1 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:41.782760 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:41.789671 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56673c18-272a-493c-a769-1a74a870b284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:41.790398 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56673c18-272a-493c-a769-1a74a870b284 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:41.790761 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56673c18-272a-493c-a769-1a74a870b284 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:41.791084 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56673c18-272a-493c-a769-1a74a870b284 None admin] PATH_INFO: `/v3/projects/695a092ba6de497f848792cc3b6bef1f` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:41.792439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56673c18-272a-493c-a769-1a74a870b284 None admin] RBAC: Authorizing `identity:get_project(project_id=695a092ba6de497f848792cc3b6bef1f)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:41.794064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56673c18-272a-493c-a769-1a74a870b284 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:41.795935 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 562/1116] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 15:00:41 2022] GET /identity/v3/projects/695a092ba6de497f848792cc3b6bef1f => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:41.802829 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fbfdc5-76ef-43b9-bc38-3c001e3aa2d0 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:41.914581 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fbfdc5-76ef-43b9-bc38-3c001e3aa2d0 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:41.950701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f7ca4-b2a8-40f3-8ea9-c701df8eb6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:41.951319 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f7ca4-b2a8-40f3-8ea9-c701df8eb6e4 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:41.951595 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f7ca4-b2a8-40f3-8ea9-c701df8eb6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:41.951865 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f7ca4-b2a8-40f3-8ea9-c701df8eb6e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:41.952595 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f7ca4-b2a8-40f3-8ea9-c701df8eb6e4 None admin] RBAC: Adding query filter params (role.id=b78105372b434d86872dd04e63dce750, scope.project.id=695a092ba6de497f848792cc3b6bef1f, user.id=89a74343be7744b7918e45599ee79b71) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:41.952897 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f7ca4-b2a8-40f3-8ea9-c701df8eb6e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:41.954386 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f7ca4-b2a8-40f3-8ea9-c701df8eb6e4 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:41.960436 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 555/1117] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 15:00:41 2022] GET /identity/v3/role_assignments?user.id=89a74343be7744b7918e45599ee79b71&scope.project.id=695a092ba6de497f848792cc3b6bef1f&role.id=b78105372b434d86872dd04e63dce750 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:44.115687 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f1ab8-1dd2-46e4-bc93-39c924fbeaef None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:44.116285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f1ab8-1dd2-46e4-bc93-39c924fbeaef None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:44.116285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f1ab8-1dd2-46e4-bc93-39c924fbeaef None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:44.117760 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 563/1118] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:44.127098 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af9848-1720-46ba-b4bd-9a5068efaf22 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:44.127674 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af9848-1720-46ba-b4bd-9a5068efaf22 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:44.128041 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af9848-1720-46ba-b4bd-9a5068efaf22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:44.157473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-39af9848-1720-46ba-b4bd-9a5068efaf22 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:44.159410 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-39af9848-1720-46ba-b4bd-9a5068efaf22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:44.239063 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:44.261969 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 556/1119] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:00:44.530826 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f1ab8-1dd2-46e4-bc93-39c924fbeaef None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:44.668154 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f1ab8-1dd2-46e4-bc93-39c924fbeaef None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:44.708538 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313d0e3-42de-473d-bc51-374d187a36dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:44.709189 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313d0e3-42de-473d-bc51-374d187a36dd None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:44.709491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313d0e3-42de-473d-bc51-374d187a36dd None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:44.709764 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313d0e3-42de-473d-bc51-374d187a36dd None admin] PATH_INFO: `/v3/roles/b78105372b434d86872dd04e63dce750` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:44.711313 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1313d0e3-42de-473d-bc51-374d187a36dd None admin] RBAC: Authorizing `identity:get_role(role_id=b78105372b434d86872dd04e63dce750)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:44.712805 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1313d0e3-42de-473d-bc51-374d187a36dd None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:44.714241 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 564/1120] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 15:00:44 2022] GET /identity/v3/roles/b78105372b434d86872dd04e63dce750 => generated 223 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:44.721273 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af9848-1720-46ba-b4bd-9a5068efaf22 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:44.844173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af9848-1720-46ba-b4bd-9a5068efaf22 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:44.885790 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411776b8-bb08-44bb-a74d-fb0ce622e8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:44.886448 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411776b8-bb08-44bb-a74d-fb0ce622e8f6 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:44.886723 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411776b8-bb08-44bb-a74d-fb0ce622e8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:44.886991 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411776b8-bb08-44bb-a74d-fb0ce622e8f6 None admin] PATH_INFO: `/v3/users/89a74343be7744b7918e45599ee79b71` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:44.888258 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411776b8-bb08-44bb-a74d-fb0ce622e8f6 None admin] RBAC: Authorizing `identity:get_user(user_id=89a74343be7744b7918e45599ee79b71)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:44.889818 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411776b8-bb08-44bb-a74d-fb0ce622e8f6 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:44.891509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 557/1121] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 15:00:44 2022] GET /identity/v3/users/89a74343be7744b7918e45599ee79b71 => generated 286 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:44.898119 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313d0e3-42de-473d-bc51-374d187a36dd None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:44.977322 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:45.019830 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313d0e3-42de-473d-bc51-374d187a36dd None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:45.057246 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f57f7-1e47-4d3b-bbc8-d9c9f5d4d6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:45.057911 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f57f7-1e47-4d3b-bbc8-d9c9f5d4d6cc None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:45.058183 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f57f7-1e47-4d3b-bbc8-d9c9f5d4d6cc None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:45.058448 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f57f7-1e47-4d3b-bbc8-d9c9f5d4d6cc None admin] PATH_INFO: `/v3/projects/695a092ba6de497f848792cc3b6bef1f` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:45.059759 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f57f7-1e47-4d3b-bbc8-d9c9f5d4d6cc None admin] RBAC: Authorizing `identity:get_project(project_id=695a092ba6de497f848792cc3b6bef1f)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:45.061294 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f57f7-1e47-4d3b-bbc8-d9c9f5d4d6cc None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:45.063097 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 565/1122] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 15:00:44 2022] GET /identity/v3/projects/695a092ba6de497f848792cc3b6bef1f => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:45.069951 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411776b8-bb08-44bb-a74d-fb0ce622e8f6 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:45.127852 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:45.185006 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411776b8-bb08-44bb-a74d-fb0ce622e8f6 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:45.238469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b14b9-7311-4656-838a-7c136b42083a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:45.239575 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b14b9-7311-4656-838a-7c136b42083a None admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:45.240045 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b14b9-7311-4656-838a-7c136b42083a None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:45.240524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b14b9-7311-4656-838a-7c136b42083a None admin] PATH_INFO: `/v3/projects/695a092ba6de497f848792cc3b6bef1f/users/89a74343be7744b7918e45599ee79b71/roles/b78105372b434d86872dd04e63dce750` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:45.244430 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b14b9-7311-4656-838a-7c136b42083a None admin] RBAC: Authorizing `identity:create_grant(project_id=695a092ba6de497f848792cc3b6bef1f, user_id=89a74343be7744b7918e45599ee79b71, role_id=b78105372b434d86872dd04e63dce750)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:45.247023 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b14b9-7311-4656-838a-7c136b42083a None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:45.265128 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 558/1123] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 15:00:45 2022] PUT /identity/v3/projects/695a092ba6de497f848792cc3b6bef1f/users/89a74343be7744b7918e45599ee79b71/roles/b78105372b434d86872dd04e63dce750 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:00:47.427137 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b0fee-113b-4f87-8927-72a1798ef254 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:47.427759 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b0fee-113b-4f87-8927-72a1798ef254 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:47.427759 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b0fee-113b-4f87-8927-72a1798ef254 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:47.428956 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 566/1124] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:47.435043 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc4aad-d99f-47f7-b0dc-590d281caa36 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:47.435590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc4aad-d99f-47f7-b0dc-590d281caa36 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:47.435776 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc4aad-d99f-47f7-b0dc-590d281caa36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:47.455696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-efcc4aad-d99f-47f7-b0dc-590d281caa36 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:47.456935 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-efcc4aad-d99f-47f7-b0dc-590d281caa36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:47.532534 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 559/1125] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:00:47.819426 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b0fee-113b-4f87-8927-72a1798ef254 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:47.942235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b0fee-113b-4f87-8927-72a1798ef254 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:47.983949 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa46aff-e9c5-4983-85bc-63ee5475f318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:47.984606 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa46aff-e9c5-4983-85bc-63ee5475f318 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:47.984883 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa46aff-e9c5-4983-85bc-63ee5475f318 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:47.985186 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa46aff-e9c5-4983-85bc-63ee5475f318 None admin] PATH_INFO: `/v3/roles/b78105372b434d86872dd04e63dce750` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:47.986730 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa46aff-e9c5-4983-85bc-63ee5475f318 None admin] RBAC: Authorizing `identity:get_role(role_id=b78105372b434d86872dd04e63dce750)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:47.988139 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa46aff-e9c5-4983-85bc-63ee5475f318 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:47.989655 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 567/1126] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 15:00:47 2022] GET /identity/v3/roles/b78105372b434d86872dd04e63dce750 => generated 223 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:47.995991 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc4aad-d99f-47f7-b0dc-590d281caa36 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:48.042362 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:48.106259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc4aad-d99f-47f7-b0dc-590d281caa36 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:48.142644 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e1dfb-1704-43c3-809b-9b76e2c6219c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:48.143298 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e1dfb-1704-43c3-809b-9b76e2c6219c None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:48.143568 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e1dfb-1704-43c3-809b-9b76e2c6219c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:48.143835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e1dfb-1704-43c3-809b-9b76e2c6219c None admin] PATH_INFO: `/v3/users/89a74343be7744b7918e45599ee79b71` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:48.145016 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e1dfb-1704-43c3-809b-9b76e2c6219c None admin] RBAC: Authorizing `identity:get_user(user_id=89a74343be7744b7918e45599ee79b71)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:48.146530 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e1dfb-1704-43c3-809b-9b76e2c6219c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:48.148167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 560/1127] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 15:00:47 2022] GET /identity/v3/users/89a74343be7744b7918e45599ee79b71 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:48.154537 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa46aff-e9c5-4983-85bc-63ee5475f318 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:48.157811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:48.267427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa46aff-e9c5-4983-85bc-63ee5475f318 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:48.304747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59402b55-e9f7-4168-bf4b-0cc5664124fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:48.305460 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59402b55-e9f7-4168-bf4b-0cc5664124fb None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:48.305792 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59402b55-e9f7-4168-bf4b-0cc5664124fb None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:48.306141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59402b55-e9f7-4168-bf4b-0cc5664124fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:48.384868 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59402b55-e9f7-4168-bf4b-0cc5664124fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:48.386283 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59402b55-e9f7-4168-bf4b-0cc5664124fb None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:48.388249 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:48.493719 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 568/1128] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 15:00:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 15:00:48.500012 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e1dfb-1704-43c3-809b-9b76e2c6219c None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:48.608066 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e1dfb-1704-43c3-809b-9b76e2c6219c None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:48.620536 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:48.649044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f01b848-fe84-4316-905e-4c28baabb94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:48.649689 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01b848-fe84-4316-905e-4c28baabb94c None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:48.650074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01b848-fe84-4316-905e-4c28baabb94c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:48.650387 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01b848-fe84-4316-905e-4c28baabb94c None admin] PATH_INFO: `/v3/projects/695a092ba6de497f848792cc3b6bef1f` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:48.651681 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f01b848-fe84-4316-905e-4c28baabb94c None admin] RBAC: Authorizing `identity:get_project(project_id=695a092ba6de497f848792cc3b6bef1f)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:48.653270 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f01b848-fe84-4316-905e-4c28baabb94c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:48.655128 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 561/1129] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 15:00:48 2022] GET /identity/v3/projects/695a092ba6de497f848792cc3b6bef1f => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:48.661937 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59402b55-e9f7-4168-bf4b-0cc5664124fb None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:48.776736 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59402b55-e9f7-4168-bf4b-0cc5664124fb None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:48.778669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:48.817275 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6b3f1-ea72-405a-8bfb-8130c480cf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:48.817913 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6b3f1-ea72-405a-8bfb-8130c480cf9e None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:48.818181 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6b3f1-ea72-405a-8bfb-8130c480cf9e None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:48.818444 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6b3f1-ea72-405a-8bfb-8130c480cf9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:48.819215 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6b3f1-ea72-405a-8bfb-8130c480cf9e None admin] RBAC: Adding query filter params (role.id=b78105372b434d86872dd04e63dce750, scope.project.id=695a092ba6de497f848792cc3b6bef1f, user.id=89a74343be7744b7918e45599ee79b71) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:48.819520 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6b3f1-ea72-405a-8bfb-8130c480cf9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:48.821036 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6b3f1-ea72-405a-8bfb-8130c480cf9e None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:48.828152 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 569/1130] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 15:00:48 2022] GET /identity/v3/role_assignments?user.id=89a74343be7744b7918e45599ee79b71&scope.project.id=695a092ba6de497f848792cc3b6bef1f&role.id=b78105372b434d86872dd04e63dce750 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:51.061960 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a92f7-5d53-4db1-8394-f502c1f6169a None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:51.062565 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a92f7-5d53-4db1-8394-f502c1f6169a None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:51.062565 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a92f7-5d53-4db1-8394-f502c1f6169a None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:51.063830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 562/1131] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:51.070007 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18a81d-6e35-49f3-8236-38be15ad6616 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:51.070806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18a81d-6e35-49f3-8236-38be15ad6616 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:51.070987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18a81d-6e35-49f3-8236-38be15ad6616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:51.093729 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-8d18a81d-6e35-49f3-8236-38be15ad6616 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:51.095336 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-8d18a81d-6e35-49f3-8236-38be15ad6616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:51.170134 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 570/1132] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:00:51.436351 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a92f7-5d53-4db1-8394-f502c1f6169a None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:51.565146 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a92f7-5d53-4db1-8394-f502c1f6169a None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:51.606908 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287e785-01d6-4360-aa7f-79064e546512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:51.607585 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287e785-01d6-4360-aa7f-79064e546512 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:51.607859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287e785-01d6-4360-aa7f-79064e546512 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:51.608123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287e785-01d6-4360-aa7f-79064e546512 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:51.609526 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287e785-01d6-4360-aa7f-79064e546512 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:51.611054 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287e785-01d6-4360-aa7f-79064e546512 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:51.612831 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 563/1133] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:00:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:51.619587 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18a81d-6e35-49f3-8236-38be15ad6616 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:51.734668 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:51.741634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18a81d-6e35-49f3-8236-38be15ad6616 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:51.781191 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d60807-bcdc-4eaf-9693-9f253a164be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:51.781893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d60807-bcdc-4eaf-9693-9f253a164be8 None admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:51.782204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d60807-bcdc-4eaf-9693-9f253a164be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:51.782509 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d60807-bcdc-4eaf-9693-9f253a164be8 None admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:51.783215 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d60807-bcdc-4eaf-9693-9f253a164be8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:51.784743 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d60807-bcdc-4eaf-9693-9f253a164be8 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:51.818338 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 571/1134] 10.222.0.44 () {60 vars in 1141 bytes} [Sun Sep 11 15:00:51 2022] POST /identity/v3/projects => generated 311 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:00:54.071106 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6982c62-08ba-4519-90d5-bb3151d08bcc None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:54.071720 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6982c62-08ba-4519-90d5-bb3151d08bcc None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:54.071720 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6982c62-08ba-4519-90d5-bb3151d08bcc None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:54.073173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 564/1135] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:54.079249 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3f5dc-b995-4144-84d6-c874a266d492 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:54.079778 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3f5dc-b995-4144-84d6-c874a266d492 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:54.079778 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3f5dc-b995-4144-84d6-c874a266d492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:54.099761 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-edf3f5dc-b995-4144-84d6-c874a266d492 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:54.101011 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-edf3f5dc-b995-4144-84d6-c874a266d492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:54.179251 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 572/1136] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:00:54.446792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6982c62-08ba-4519-90d5-bb3151d08bcc None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:54.498911 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:54.559408 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6982c62-08ba-4519-90d5-bb3151d08bcc None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:54.598521 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9556bd-0f54-4e5e-99fe-bb1d078c44a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:54.599195 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9556bd-0f54-4e5e-99fe-bb1d078c44a1 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:54.599465 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9556bd-0f54-4e5e-99fe-bb1d078c44a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:54.599734 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9556bd-0f54-4e5e-99fe-bb1d078c44a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:54.600940 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9556bd-0f54-4e5e-99fe-bb1d078c44a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:54.602458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9556bd-0f54-4e5e-99fe-bb1d078c44a1 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:54.604264 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 565/1137] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:00:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:54.611140 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3f5dc-b995-4144-84d6-c874a266d492 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:54.755630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:54.798033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3f5dc-b995-4144-84d6-c874a266d492 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:54.859006 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da5b6d-7bf5-4aaa-b03d-3d98d53d7bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:54.859929 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da5b6d-7bf5-4aaa-b03d-3d98d53d7bfc None admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:54.860359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da5b6d-7bf5-4aaa-b03d-3d98d53d7bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:54.860821 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da5b6d-7bf5-4aaa-b03d-3d98d53d7bfc None admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:54.862032 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da5b6d-7bf5-4aaa-b03d-3d98d53d7bfc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:54.864712 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da5b6d-7bf5-4aaa-b03d-3d98d53d7bfc None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:54.894816 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 573/1138] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 15:00:54 2022] POST /identity/v3/users => generated 286 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:00:57.127500 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5aebc2-01a8-49f5-9342-e1da23353865 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:57.128383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5aebc2-01a8-49f5-9342-e1da23353865 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:57.128383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5aebc2-01a8-49f5-9342-e1da23353865 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:57.129603 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 566/1139] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:00:57.136657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2970db0-18a6-4f22-9d49-acfb677b2964 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:57.137316 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2970db0-18a6-4f22-9d49-acfb677b2964 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:57.137316 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2970db0-18a6-4f22-9d49-acfb677b2964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:57.160223 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-b2970db0-18a6-4f22-9d49-acfb677b2964 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:00:57.162442 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-b2970db0-18a6-4f22-9d49-acfb677b2964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:00:57.275260 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 574/1140] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:00:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:00:57.543641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5aebc2-01a8-49f5-9342-e1da23353865 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:57.667191 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5aebc2-01a8-49f5-9342-e1da23353865 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:57.683208 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:57.704541 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5442bb-b84c-40ef-aa3c-caabc6f2732f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:57.705203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5442bb-b84c-40ef-aa3c-caabc6f2732f None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:57.705480 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5442bb-b84c-40ef-aa3c-caabc6f2732f None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:57.705744 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5442bb-b84c-40ef-aa3c-caabc6f2732f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:57.716879 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5442bb-b84c-40ef-aa3c-caabc6f2732f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 15:00:57.722074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-cb5442bb-b84c-40ef-aa3c-caabc6f2732f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 15:00:57.722074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:00:57.722074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:00:57.722074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:00:57.722074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:00:57.722074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:00:57.722074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:00:57.722074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:00:57.722074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:00:57.722074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:00:57.722074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:00:57.722074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:00:57.722074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 15:00:57.722074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 15:00:57.722074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 15:00:57.722074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 15:00:57.723284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 15:00:57.723284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 15:00:57.723284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 15:00:57.723284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 15:00:57.723284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:00:57.723284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:00:57.723284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:00:57.723284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:00:57.723284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:00:57.723284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:00:57.723284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:00:57.723284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:00:57.723284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:00:57.723284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:00:57.723284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:00:57.723284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:00:57.724248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:00:57.724248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:00:57.724248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:00:57.724248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:00:57.724248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 15:00:57.724248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 15:00:57.724248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 15:00:57.724248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 15:00:57.724248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 15:00:57.724248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 15:00:57.724248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 15:00:57.724248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:00:57.724248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 567/1141] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 15:00:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:00:57.729813 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2970db0-18a6-4f22-9d49-acfb677b2964 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:57.797616 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:57.840042 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2970db0-18a6-4f22-9d49-acfb677b2964 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:57.877203 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22946e8c-17fd-4b26-84da-cc9aab8d2d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:57.877880 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22946e8c-17fd-4b26-84da-cc9aab8d2d00 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:57.878194 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22946e8c-17fd-4b26-84da-cc9aab8d2d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:57.878506 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22946e8c-17fd-4b26-84da-cc9aab8d2d00 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:57.879211 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22946e8c-17fd-4b26-84da-cc9aab8d2d00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:57.879545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22946e8c-17fd-4b26-84da-cc9aab8d2d00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:57.880963 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22946e8c-17fd-4b26-84da-cc9aab8d2d00 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:57.890306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 575/1142] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 15:00:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:00:57.897304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5442bb-b84c-40ef-aa3c-caabc6f2732f None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:58.071312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5442bb-b84c-40ef-aa3c-caabc6f2732f None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:58.130929 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e994124b-5e62-47c5-9d55-2666d7753dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:58.131996 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994124b-5e62-47c5-9d55-2666d7753dbe None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:58.132468 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994124b-5e62-47c5-9d55-2666d7753dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:58.132917 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994124b-5e62-47c5-9d55-2666d7753dbe None admin] PATH_INFO: `/v3/users/f4b29508a602477282b6cc1cbb89d8ce` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:58.158364 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e994124b-5e62-47c5-9d55-2666d7753dbe None admin] RBAC: Authorizing `identity:get_user(user_id=f4b29508a602477282b6cc1cbb89d8ce)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:58.161079 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e994124b-5e62-47c5-9d55-2666d7753dbe None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:58.164241 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 568/1143] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 15:00:57 2022] GET /identity/v3/users/f4b29508a602477282b6cc1cbb89d8ce => generated 286 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:58.170916 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22946e8c-17fd-4b26-84da-cc9aab8d2d00 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:58.292645 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22946e8c-17fd-4b26-84da-cc9aab8d2d00 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:58.323388 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:58.337969 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9218547-0369-4f2c-9a8c-71a5f86a71b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:58.338977 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9218547-0369-4f2c-9a8c-71a5f86a71b6 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:58.339458 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9218547-0369-4f2c-9a8c-71a5f86a71b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:58.339922 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9218547-0369-4f2c-9a8c-71a5f86a71b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:58.458292 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9218547-0369-4f2c-9a8c-71a5f86a71b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:58.460976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9218547-0369-4f2c-9a8c-71a5f86a71b6 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:58.639940 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 576/1144] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 15:00:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 15:00:58.646297 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e994124b-5e62-47c5-9d55-2666d7753dbe None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:58.681639 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:58.756259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e994124b-5e62-47c5-9d55-2666d7753dbe None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:58.794807 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5673cd43-76a5-4c1c-b675-256c5c81f26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:58.795476 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673cd43-76a5-4c1c-b675-256c5c81f26c None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:58.795793 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673cd43-76a5-4c1c-b675-256c5c81f26c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:58.796114 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673cd43-76a5-4c1c-b675-256c5c81f26c None admin] PATH_INFO: `/v3/projects/b647e210cdf24086b06573f9fb6cd9aa` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:58.797366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5673cd43-76a5-4c1c-b675-256c5c81f26c None admin] RBAC: Authorizing `identity:get_project(project_id=b647e210cdf24086b06573f9fb6cd9aa)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:58.798857 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5673cd43-76a5-4c1c-b675-256c5c81f26c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:58.800598 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 569/1145] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 15:00:58 2022] GET /identity/v3/projects/b647e210cdf24086b06573f9fb6cd9aa => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:00:58.807687 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9218547-0369-4f2c-9a8c-71a5f86a71b6 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:00:58.852780 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:00:58.977445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9218547-0369-4f2c-9a8c-71a5f86a71b6 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:00:59.033837 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b19438-9386-480f-9528-c6981eef51ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:00:59.034859 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b19438-9386-480f-9528-c6981eef51ad None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:00:59.035319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b19438-9386-480f-9528-c6981eef51ad None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:00:59.035777 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b19438-9386-480f-9528-c6981eef51ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:00:59.037095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b19438-9386-480f-9528-c6981eef51ad None admin] RBAC: Adding query filter params (role.id=c426875a15834aa5b122e2a62267d15a, scope.project.id=b647e210cdf24086b06573f9fb6cd9aa, user.id=f4b29508a602477282b6cc1cbb89d8ce) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:00:59.037655 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b19438-9386-480f-9528-c6981eef51ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:00:59.040258 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b19438-9386-480f-9528-c6981eef51ad None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:00:59.050162 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 577/1146] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 15:00:58 2022] GET /identity/v3/role_assignments?user.id=f4b29508a602477282b6cc1cbb89d8ce&scope.project.id=b647e210cdf24086b06573f9fb6cd9aa&role.id=c426875a15834aa5b122e2a62267d15a => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:01:01.260399 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54fe3f-e39c-42bb-9247-e17aea6b6141 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:01.261044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54fe3f-e39c-42bb-9247-e17aea6b6141 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:01.261044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54fe3f-e39c-42bb-9247-e17aea6b6141 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:01.262316 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 570/1147] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:01.268279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624246ae-1612-4066-81fe-99cd1803b2d5 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:01.268840 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624246ae-1612-4066-81fe-99cd1803b2d5 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:01.268840 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624246ae-1612-4066-81fe-99cd1803b2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:01.288450 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-624246ae-1612-4066-81fe-99cd1803b2d5 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:01.289792 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-624246ae-1612-4066-81fe-99cd1803b2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:01.364181 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 578/1148] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:01:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:01:01.629340 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54fe3f-e39c-42bb-9247-e17aea6b6141 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:01.747219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:01.752144 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54fe3f-e39c-42bb-9247-e17aea6b6141 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:01.795808 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6eca50-0648-4d36-b1ad-7ec94632640a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:01.796599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6eca50-0648-4d36-b1ad-7ec94632640a None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:01.796956 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6eca50-0648-4d36-b1ad-7ec94632640a None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:01.797425 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6eca50-0648-4d36-b1ad-7ec94632640a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:01.810503 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6eca50-0648-4d36-b1ad-7ec94632640a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 15:01:01.815419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-0d6eca50-0648-4d36-b1ad-7ec94632640a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 15:01:01.815419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:01:01.815419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:01:01.815419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:01:01.815419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:01:01.815419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:01:01.815419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:01:01.815419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:01:01.815419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:01:01.815419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:01:01.815419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:01:01.815419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:01:01.815419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 15:01:01.815419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 15:01:01.815419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 15:01:01.815419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 15:01:01.816802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 15:01:01.816802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 15:01:01.816802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 15:01:01.816802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 15:01:01.816802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:01:01.816802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:01:01.816802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:01:01.816802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:01:01.816802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:01:01.816802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:01:01.816802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:01:01.816802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:01:01.816802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:01:01.816802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:01:01.816802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:01:01.816802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:01:01.818928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:01:01.818928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:01:01.818928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:01:01.818928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:01:01.818928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 15:01:01.818928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 15:01:01.818928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 15:01:01.818928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 15:01:01.818928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 15:01:01.818928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 15:01:01.818928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 15:01:01.818928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:01:01.818928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 571/1149] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 15:01:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:01:01.825305 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624246ae-1612-4066-81fe-99cd1803b2d5 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:01.851031 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:01.936696 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624246ae-1612-4066-81fe-99cd1803b2d5 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:01.974566 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f70a2b8-4fea-416b-b0a9-483e23ecc066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:01.975183 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70a2b8-4fea-416b-b0a9-483e23ecc066 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:01.975424 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70a2b8-4fea-416b-b0a9-483e23ecc066 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:01.975698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70a2b8-4fea-416b-b0a9-483e23ecc066 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:01.976352 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f70a2b8-4fea-416b-b0a9-483e23ecc066 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:01:01.976656 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f70a2b8-4fea-416b-b0a9-483e23ecc066 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:01.978125 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f70a2b8-4fea-416b-b0a9-483e23ecc066 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:01.987317 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 579/1150] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 15:01:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:01:01.993467 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6eca50-0648-4d36-b1ad-7ec94632640a None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:02.101703 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6eca50-0648-4d36-b1ad-7ec94632640a None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:02.145105 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bcdaf0-411d-4ba8-9e18-d62af7c1aaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:02.145781 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcdaf0-411d-4ba8-9e18-d62af7c1aaa8 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:02.146054 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcdaf0-411d-4ba8-9e18-d62af7c1aaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:02.146313 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcdaf0-411d-4ba8-9e18-d62af7c1aaa8 None admin] PATH_INFO: `/v3/users/f4b29508a602477282b6cc1cbb89d8ce` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:02.147453 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bcdaf0-411d-4ba8-9e18-d62af7c1aaa8 None admin] RBAC: Authorizing `identity:get_user(user_id=f4b29508a602477282b6cc1cbb89d8ce)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:02.148955 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bcdaf0-411d-4ba8-9e18-d62af7c1aaa8 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:02.150779 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 572/1151] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 15:01:01 2022] GET /identity/v3/users/f4b29508a602477282b6cc1cbb89d8ce => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:01:02.157039 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f70a2b8-4fea-416b-b0a9-483e23ecc066 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:02.245393 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:02.267813 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f70a2b8-4fea-416b-b0a9-483e23ecc066 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:02.305893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf7869-0218-488a-ba82-4ae90fb70849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:02.306579 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf7869-0218-488a-ba82-4ae90fb70849 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:02.306808 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf7869-0218-488a-ba82-4ae90fb70849 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:02.307080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf7869-0218-488a-ba82-4ae90fb70849 None admin] PATH_INFO: `/v3/projects/b647e210cdf24086b06573f9fb6cd9aa` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:02.308329 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf7869-0218-488a-ba82-4ae90fb70849 None admin] RBAC: Authorizing `identity:get_project(project_id=b647e210cdf24086b06573f9fb6cd9aa)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:02.309838 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf7869-0218-488a-ba82-4ae90fb70849 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:02.311558 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 580/1152] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 15:01:02 2022] GET /identity/v3/projects/b647e210cdf24086b06573f9fb6cd9aa => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:01:02.318011 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bcdaf0-411d-4ba8-9e18-d62af7c1aaa8 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:02.349356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:02.423514 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bcdaf0-411d-4ba8-9e18-d62af7c1aaa8 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:02.461617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac718b-07a2-429b-9b26-2deaea8d5e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:02.462362 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac718b-07a2-429b-9b26-2deaea8d5e84 None admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:02.462627 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac718b-07a2-429b-9b26-2deaea8d5e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:02.462892 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac718b-07a2-429b-9b26-2deaea8d5e84 None admin] PATH_INFO: `/v3/projects/b647e210cdf24086b06573f9fb6cd9aa/users/f4b29508a602477282b6cc1cbb89d8ce/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:02.464891 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac718b-07a2-429b-9b26-2deaea8d5e84 None admin] RBAC: Authorizing `identity:create_grant(project_id=b647e210cdf24086b06573f9fb6cd9aa, user_id=f4b29508a602477282b6cc1cbb89d8ce, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:02.466366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac718b-07a2-429b-9b26-2deaea8d5e84 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:02.482622 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 573/1153] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 15:01:02 2022] PUT /identity/v3/projects/b647e210cdf24086b06573f9fb6cd9aa/users/f4b29508a602477282b6cc1cbb89d8ce/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:01:04.653654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f73eb-fdf6-4ce8-a3e0-f880591faae8 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:04.654525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f73eb-fdf6-4ce8-a3e0-f880591faae8 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:04.654525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f73eb-fdf6-4ce8-a3e0-f880591faae8 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:04.655408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 581/1154] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 15:01:04.661226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad12c9-8cb3-4f58-bfae-7552fe5ced20 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:04.661708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad12c9-8cb3-4f58-bfae-7552fe5ced20 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:04.661908 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad12c9-8cb3-4f58-bfae-7552fe5ced20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:04.681573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-81ad12c9-8cb3-4f58-bfae-7552fe5ced20 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:04.682795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-81ad12c9-8cb3-4f58-bfae-7552fe5ced20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:04.755400 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 574/1155] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:01:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:01:05.021921 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f73eb-fdf6-4ce8-a3e0-f880591faae8 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:05.142819 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f73eb-fdf6-4ce8-a3e0-f880591faae8 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:05.184394 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea95b0-6ce1-4be3-b206-4fc0489b5f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:05.185045 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea95b0-6ce1-4be3-b206-4fc0489b5f78 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:05.185408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea95b0-6ce1-4be3-b206-4fc0489b5f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:05.185580 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea95b0-6ce1-4be3-b206-4fc0489b5f78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:05.198071 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ea95b0-6ce1-4be3-b206-4fc0489b5f78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 15:01:05.202918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-15ea95b0-6ce1-4be3-b206-4fc0489b5f78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 15:01:05.202918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:01:05.202918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:01:05.202918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:01:05.202918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:01:05.202918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:01:05.202918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:01:05.202918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:01:05.202918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:01:05.202918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:01:05.202918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:01:05.202918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:01:05.202918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 15:01:05.202918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 15:01:05.202918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 15:01:05.202918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 15:01:05.204688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 15:01:05.204688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 15:01:05.204688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 15:01:05.204688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 15:01:05.204688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:01:05.204688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:01:05.204688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:01:05.204688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:01:05.204688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:01:05.204688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:01:05.204688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:01:05.204688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:01:05.204688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:01:05.204688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:01:05.204688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:01:05.204688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:01:05.206125 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:01:05.206125 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:01:05.206125 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:01:05.206125 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:01:05.206125 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 15:01:05.206125 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 15:01:05.206125 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 15:01:05.206125 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 15:01:05.206125 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 15:01:05.206125 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 15:01:05.206125 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 15:01:05.206125 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:01:05.206125 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 582/1156] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 15:01:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:01:05.213394 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad12c9-8cb3-4f58-bfae-7552fe5ced20 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:05.239823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:05.346493 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad12c9-8cb3-4f58-bfae-7552fe5ced20 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:05.387640 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67bd4e2-51e0-485f-9f04-6b314e885362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:05.388281 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67bd4e2-51e0-485f-9f04-6b314e885362 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:05.388582 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67bd4e2-51e0-485f-9f04-6b314e885362 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:05.388837 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67bd4e2-51e0-485f-9f04-6b314e885362 None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:05.389528 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67bd4e2-51e0-485f-9f04-6b314e885362 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:01:05.389833 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67bd4e2-51e0-485f-9f04-6b314e885362 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:05.391250 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67bd4e2-51e0-485f-9f04-6b314e885362 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:05.401612 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 575/1157] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 15:01:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:01:05.408150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea95b0-6ce1-4be3-b206-4fc0489b5f78 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:05.422828 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:05.525658 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea95b0-6ce1-4be3-b206-4fc0489b5f78 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:05.577432 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3429597-f580-4960-9333-d988056c8597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:05.577867 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3429597-f580-4960-9333-d988056c8597 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:05.578187 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3429597-f580-4960-9333-d988056c8597 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:05.578500 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3429597-f580-4960-9333-d988056c8597 None admin] PATH_INFO: `/v3/users/f4b29508a602477282b6cc1cbb89d8ce` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:05.580141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3429597-f580-4960-9333-d988056c8597 None admin] RBAC: Authorizing `identity:get_user(user_id=f4b29508a602477282b6cc1cbb89d8ce)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:05.582338 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3429597-f580-4960-9333-d988056c8597 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:05.584036 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 583/1158] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 15:01:05 2022] GET /identity/v3/users/f4b29508a602477282b6cc1cbb89d8ce => generated 286 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:01:05.590402 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67bd4e2-51e0-485f-9f04-6b314e885362 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:05.682667 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:05.704450 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67bd4e2-51e0-485f-9f04-6b314e885362 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:05.741497 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a2720-abce-4ec0-bc9b-a11a4fb7a410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:05.742125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a2720-abce-4ec0-bc9b-a11a4fb7a410 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:05.742406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a2720-abce-4ec0-bc9b-a11a4fb7a410 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:05.742675 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a2720-abce-4ec0-bc9b-a11a4fb7a410 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:05.814497 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a2720-abce-4ec0-bc9b-a11a4fb7a410 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:05.816052 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a2720-abce-4ec0-bc9b-a11a4fb7a410 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:05.919508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:05.954203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 576/1159] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 15:01:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 15:01:05.960694 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3429597-f580-4960-9333-d988056c8597 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:06.044894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:06.069628 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3429597-f580-4960-9333-d988056c8597 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:06.108599 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12951dc-1240-4f27-bdea-e89a8ab642cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:06.109347 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12951dc-1240-4f27-bdea-e89a8ab642cb None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:06.109667 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12951dc-1240-4f27-bdea-e89a8ab642cb None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:06.109974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12951dc-1240-4f27-bdea-e89a8ab642cb None admin] PATH_INFO: `/v3/projects/b647e210cdf24086b06573f9fb6cd9aa` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:06.111365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12951dc-1240-4f27-bdea-e89a8ab642cb None admin] RBAC: Authorizing `identity:get_project(project_id=b647e210cdf24086b06573f9fb6cd9aa)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:06.112962 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12951dc-1240-4f27-bdea-e89a8ab642cb None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:06.114857 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 584/1160] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 15:01:05 2022] GET /identity/v3/projects/b647e210cdf24086b06573f9fb6cd9aa => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:01:06.122021 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a2720-abce-4ec0-bc9b-a11a4fb7a410 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:06.237182 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a2720-abce-4ec0-bc9b-a11a4fb7a410 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:06.279212 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893b800-7720-41a3-becc-59ec0748c761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:06.279877 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893b800-7720-41a3-becc-59ec0748c761 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:06.280193 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893b800-7720-41a3-becc-59ec0748c761 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:06.280500 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893b800-7720-41a3-becc-59ec0748c761 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:06.281326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893b800-7720-41a3-becc-59ec0748c761 None admin] RBAC: Adding query filter params (role.id=c426875a15834aa5b122e2a62267d15a, scope.project.id=b647e210cdf24086b06573f9fb6cd9aa, user.id=f4b29508a602477282b6cc1cbb89d8ce) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:01:06.281675 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893b800-7720-41a3-becc-59ec0748c761 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:06.283169 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893b800-7720-41a3-becc-59ec0748c761 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:06.290076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 577/1161] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 15:01:06 2022] GET /identity/v3/role_assignments?user.id=f4b29508a602477282b6cc1cbb89d8ce&scope.project.id=b647e210cdf24086b06573f9fb6cd9aa&role.id=c426875a15834aa5b122e2a62267d15a => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:01:08.604363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677b8f7-f4dd-4ac0-9502-b53c944d32e8 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:08.604956 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677b8f7-f4dd-4ac0-9502-b53c944d32e8 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:08.605267 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677b8f7-f4dd-4ac0-9502-b53c944d32e8 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:08.607703 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 585/1162] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:01:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:08.617090 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c2bc5-1450-497d-bc6e-894300a9fe74 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:08.617684 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c2bc5-1450-497d-bc6e-894300a9fe74 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:08.618060 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c2bc5-1450-497d-bc6e-894300a9fe74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:08.647686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-6f7c2bc5-1450-497d-bc6e-894300a9fe74 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:08.649698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-6f7c2bc5-1450-497d-bc6e-894300a9fe74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:08.700970 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:08.744072 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 578/1163] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:01:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:01:09.013068 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a677b8f7-f4dd-4ac0-9502-b53c944d32e8 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:09.128839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a677b8f7-f4dd-4ac0-9502-b53c944d32e8 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:09.169700 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4a5b4-7ebc-4f25-aef4-7d0bad7a0d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:09.170308 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4a5b4-7ebc-4f25-aef4-7d0bad7a0d17 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:09.170582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4a5b4-7ebc-4f25-aef4-7d0bad7a0d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:09.170851 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4a5b4-7ebc-4f25-aef4-7d0bad7a0d17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:09.190547 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:09.253663 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4a5b4-7ebc-4f25-aef4-7d0bad7a0d17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:09.255165 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4a5b4-7ebc-4f25-aef4-7d0bad7a0d17 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:09.362748 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 586/1164] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 15:01:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 15:01:09.369003 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c2bc5-1450-497d-bc6e-894300a9fe74 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:09.472133 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c2bc5-1450-497d-bc6e-894300a9fe74 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:09.508098 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901faab-837a-4e2d-a636-77613d416aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:09.508729 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901faab-837a-4e2d-a636-77613d416aab None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:09.509074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901faab-837a-4e2d-a636-77613d416aab None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:09.509370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901faab-837a-4e2d-a636-77613d416aab None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:09.516397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a901faab-837a-4e2d-a636-77613d416aab None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:09.517874 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a901faab-837a-4e2d-a636-77613d416aab None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:09.529375 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-a901faab-837a-4e2d-a636-77613d416aab None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 11 15:01:09.529375 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:01:09.529375 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 11 15:01:09.529375 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 11 15:01:09.529375 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 15:01:09.529375 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 15:01:09.529375 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 15:01:09.529375 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 15:01:09.529375 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 11 15:01:09.529375 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:01:09.529375 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 15:01:09.529375 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:01:09.529375 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:01:09.529375 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:01:09.529375 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:01:09.529375 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:01:09.529375 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:01:09.531392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:01:09.531392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:01:09.531392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:01:09.531392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:01:09.531392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:01:09.531392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:01:09.531392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 15:01:09.531392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 15:01:09.531392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 15:01:09.531392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 15:01:09.531392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:01:09.531392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:01:09.531392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:01:09.531392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:01:09.531392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:01:09.531392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:01:09.533391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:01:09.533391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:01:09.533391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:01:09.533391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:01:09.533391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:01:09.533391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:01:09.533391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:01:09.533391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:01:09.533391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:01:09.533391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:01:09.533391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 11 15:01:09.533391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 15:01:09.533391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 11 15:01:09.533391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:01:09.533391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 579/1165] 10.222.0.44 () {56 vars in 1117 bytes} [Sun Sep 11 15:01:09 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:01:09.538890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4a5b4-7ebc-4f25-aef4-7d0bad7a0d17 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:09.588671 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:09.646526 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4a5b4-7ebc-4f25-aef4-7d0bad7a0d17 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:09.682725 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a5b21a-e210-4ca0-9bc6-a171ca0cd920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:09.683339 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5b21a-e210-4ca0-9bc6-a171ca0cd920 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:09.683610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5b21a-e210-4ca0-9bc6-a171ca0cd920 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:09.683876 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5b21a-e210-4ca0-9bc6-a171ca0cd920 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:09.684522 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a5b21a-e210-4ca0-9bc6-a171ca0cd920 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:01:09.684820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a5b21a-e210-4ca0-9bc6-a171ca0cd920 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:09.686233 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a5b21a-e210-4ca0-9bc6-a171ca0cd920 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:09.693905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 587/1166] 10.222.0.44 () {56 vars in 1115 bytes} [Sun Sep 11 15:01:09 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:01:11.881342 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6547a4-ec63-47eb-93c7-3bbb30aa485a None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:11.881996 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6547a4-ec63-47eb-93c7-3bbb30aa485a None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:11.881996 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6547a4-ec63-47eb-93c7-3bbb30aa485a None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:11.883256 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 580/1167] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:11.889838 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d4c52-7e31-4ef9-9fda-90187ed8ba91 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:11.890476 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d4c52-7e31-4ef9-9fda-90187ed8ba91 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:11.890476 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d4c52-7e31-4ef9-9fda-90187ed8ba91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:11.911560 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-fc3d4c52-7e31-4ef9-9fda-90187ed8ba91 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:11.912931 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-fc3d4c52-7e31-4ef9-9fda-90187ed8ba91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:12.004084 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 588/1168] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:01:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:01:12.277112 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6547a4-ec63-47eb-93c7-3bbb30aa485a None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:12.318174 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:12.392605 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6547a4-ec63-47eb-93c7-3bbb30aa485a None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:12.430088 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be5146-36a1-4d93-b110-38936cc9122c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:12.430751 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be5146-36a1-4d93-b110-38936cc9122c None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:12.431026 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be5146-36a1-4d93-b110-38936cc9122c None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:12.431297 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be5146-36a1-4d93-b110-38936cc9122c None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:12.431970 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16be5146-36a1-4d93-b110-38936cc9122c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:12.433383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16be5146-36a1-4d93-b110-38936cc9122c None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:12.451326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 581/1169] 10.222.0.44 () {60 vars in 1138 bytes} [Sun Sep 11 15:01:12 2022] POST /identity/v3/domains => generated 267 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:01:14.750485 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c08825-31ef-4e23-84c6-e3e15adab31c None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:14.751065 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c08825-31ef-4e23-84c6-e3e15adab31c None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:14.751065 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c08825-31ef-4e23-84c6-e3e15adab31c None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:14.752252 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 589/1170] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:01:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:14.759111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79da50-e6cc-4f88-a0e9-c9c0b4d597a2 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:14.759684 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79da50-e6cc-4f88-a0e9-c9c0b4d597a2 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:14.759684 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79da50-e6cc-4f88-a0e9-c9c0b4d597a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:14.778227 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-ff79da50-e6cc-4f88-a0e9-c9c0b4d597a2 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:14.779432 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-ff79da50-e6cc-4f88-a0e9-c9c0b4d597a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:14.852644 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 582/1171] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:01:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:01:15.118752 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c08825-31ef-4e23-84c6-e3e15adab31c None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:15.169394 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:15.255244 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c08825-31ef-4e23-84c6-e3e15adab31c None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:15.298476 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f2ace-b018-4799-86b7-9b9a4d234f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:15.299168 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f2ace-b018-4799-86b7-9b9a4d234f32 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:15.299488 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f2ace-b018-4799-86b7-9b9a4d234f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:15.299797 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f2ace-b018-4799-86b7-9b9a4d234f32 None admin] PATH_INFO: `/v3/domains/00f1752393df4516bbae6030857a39b7` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:15.311686 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22f2ace-b018-4799-86b7-9b9a4d234f32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=00f1752393df4516bbae6030857a39b7)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:15.313246 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22f2ace-b018-4799-86b7-9b9a4d234f32 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:15.314965 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 590/1172] 10.222.0.44 () {56 vars in 1183 bytes} [Sun Sep 11 15:01:15 2022] GET /identity/v3/domains/00f1752393df4516bbae6030857a39b7 => generated 267 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:01:15.321546 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79da50-e6cc-4f88-a0e9-c9c0b4d597a2 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:15.349543 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:15.446946 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79da50-e6cc-4f88-a0e9-c9c0b4d597a2 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:15.484011 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb7da0-480c-4c04-97bc-4a74d7c49af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:15.484631 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb7da0-480c-4c04-97bc-4a74d7c49af6 None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:15.484910 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb7da0-480c-4c04-97bc-4a74d7c49af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:15.485177 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb7da0-480c-4c04-97bc-4a74d7c49af6 None admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:15.485866 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdb7da0-480c-4c04-97bc-4a74d7c49af6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:15.487318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdb7da0-480c-4c04-97bc-4a74d7c49af6 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:15.517782 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 583/1173] 10.222.0.44 () {60 vars in 1141 bytes} [Sun Sep 11 15:01:15 2022] POST /identity/v3/projects => generated 361 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:01:17.788638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f214490-8273-4933-8272-0dc6bf22d914 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:17.789275 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f214490-8273-4933-8272-0dc6bf22d914 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:17.789275 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f214490-8273-4933-8272-0dc6bf22d914 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:17.790804 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 591/1174] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:01:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:17.800360 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d79b6-5edd-430e-83df-88f73a2bda4e None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:17.800903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d79b6-5edd-430e-83df-88f73a2bda4e None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:17.801336 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d79b6-5edd-430e-83df-88f73a2bda4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:17.832458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-fb3d79b6-5edd-430e-83df-88f73a2bda4e None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:17.834558 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-fb3d79b6-5edd-430e-83df-88f73a2bda4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:17.957586 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 584/1175] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:01:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:01:18.452178 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f214490-8273-4933-8272-0dc6bf22d914 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:18.562560 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:18.569558 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f214490-8273-4933-8272-0dc6bf22d914 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:18.624074 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137dc58c-e04b-4b16-8f0f-38b76c0470e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:18.624738 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137dc58c-e04b-4b16-8f0f-38b76c0470e1 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:18.625009 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137dc58c-e04b-4b16-8f0f-38b76c0470e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:18.625314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137dc58c-e04b-4b16-8f0f-38b76c0470e1 None admin] PATH_INFO: `/v3/domains/00f1752393df4516bbae6030857a39b7` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:18.626539 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137dc58c-e04b-4b16-8f0f-38b76c0470e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=00f1752393df4516bbae6030857a39b7)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:18.628035 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137dc58c-e04b-4b16-8f0f-38b76c0470e1 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:18.629916 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 592/1176] 10.222.0.44 () {56 vars in 1183 bytes} [Sun Sep 11 15:01:18 2022] GET /identity/v3/domains/00f1752393df4516bbae6030857a39b7 => generated 267 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:01:18.636540 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d79b6-5edd-430e-83df-88f73a2bda4e None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:18.645676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:18.752333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d79b6-5edd-430e-83df-88f73a2bda4e None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:18.789450 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38690771-1b38-41a7-9295-7376c0c24cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:18.790113 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38690771-1b38-41a7-9295-7376c0c24cfd None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:18.790385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38690771-1b38-41a7-9295-7376c0c24cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:18.790648 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38690771-1b38-41a7-9295-7376c0c24cfd None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:18.791302 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38690771-1b38-41a7-9295-7376c0c24cfd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:18.792756 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38690771-1b38-41a7-9295-7376c0c24cfd None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:18.817698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 585/1177] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 15:01:18 2022] POST /identity/v3/users => generated 311 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:01:21.111352 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbf10a-9a52-4fef-995e-ab8fa5e6bf2e None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:21.112221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbf10a-9a52-4fef-995e-ab8fa5e6bf2e None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:21.112221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbf10a-9a52-4fef-995e-ab8fa5e6bf2e None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:21.113796 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 593/1178] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:01:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:21.119791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10333db-fef9-4972-aa76-0ed0fa3edd85 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:21.120296 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10333db-fef9-4972-aa76-0ed0fa3edd85 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:21.120296 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10333db-fef9-4972-aa76-0ed0fa3edd85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:21.139228 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-f10333db-fef9-4972-aa76-0ed0fa3edd85 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:21.140434 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f10333db-fef9-4972-aa76-0ed0fa3edd85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:21.201794 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:21.215277 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 586/1179] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:01:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:01:21.506721 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbbf10a-9a52-4fef-995e-ab8fa5e6bf2e None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:21.620023 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbbf10a-9a52-4fef-995e-ab8fa5e6bf2e None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:21.656459 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78315fa-1826-4b6c-b65e-1c63f03feaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:21.657109 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78315fa-1826-4b6c-b65e-1c63f03feaae None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:21.657420 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78315fa-1826-4b6c-b65e-1c63f03feaae None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:21.657736 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78315fa-1826-4b6c-b65e-1c63f03feaae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:21.670338 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78315fa-1826-4b6c-b65e-1c63f03feaae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 15:01:21.675204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-e78315fa-1826-4b6c-b65e-1c63f03feaae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 15:01:21.675204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:01:21.675204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:01:21.675204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:01:21.675204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:01:21.675204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:01:21.675204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:01:21.675204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:01:21.675204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:01:21.675204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:01:21.675204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:01:21.675204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:01:21.675204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 15:01:21.675204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 15:01:21.675204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 15:01:21.675204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 15:01:21.676357 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 15:01:21.676357 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 15:01:21.676357 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 15:01:21.676357 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 15:01:21.676357 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:01:21.676357 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:01:21.676357 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:01:21.676357 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:01:21.676357 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:01:21.676357 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:01:21.676357 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:01:21.676357 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:01:21.676357 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:01:21.676357 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:01:21.676357 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:01:21.676357 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:01:21.677423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:01:21.677423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:01:21.677423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:01:21.677423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:01:21.677423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 15:01:21.677423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 15:01:21.677423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 15:01:21.677423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 15:01:21.677423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 15:01:21.677423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 15:01:21.677423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 15:01:21.677423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:01:21.677423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 594/1180] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 15:01:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:01:21.682782 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10333db-fef9-4972-aa76-0ed0fa3edd85 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:21.790823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10333db-fef9-4972-aa76-0ed0fa3edd85 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:21.827408 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6f1aa-254b-4fb8-9014-00abda6d0fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:21.828015 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6f1aa-254b-4fb8-9014-00abda6d0fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:21.828284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6f1aa-254b-4fb8-9014-00abda6d0fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:21.828601 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6f1aa-254b-4fb8-9014-00abda6d0fb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:21.829295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe6f1aa-254b-4fb8-9014-00abda6d0fb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:01:21.829590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe6f1aa-254b-4fb8-9014-00abda6d0fb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:21.830997 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe6f1aa-254b-4fb8-9014-00abda6d0fb0 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:21.840120 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 587/1181] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 15:01:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:01:21.849919 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78315fa-1826-4b6c-b65e-1c63f03feaae None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:21.884635 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:21.965279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78315fa-1826-4b6c-b65e-1c63f03feaae None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:22.005309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97fb6a-180b-40e4-b23e-e5c6ff718302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:22.005976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97fb6a-180b-40e4-b23e-e5c6ff718302 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:22.006248 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97fb6a-180b-40e4-b23e-e5c6ff718302 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:22.006516 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97fb6a-180b-40e4-b23e-e5c6ff718302 None admin] PATH_INFO: `/v3/users/47d28b15d8b3475a812012d92446a7e1` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:22.022844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97fb6a-180b-40e4-b23e-e5c6ff718302 None admin] RBAC: Authorizing `identity:get_user(user_id=47d28b15d8b3475a812012d92446a7e1)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:22.024349 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97fb6a-180b-40e4-b23e-e5c6ff718302 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:22.026162 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 595/1182] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 15:01:21 2022] GET /identity/v3/users/47d28b15d8b3475a812012d92446a7e1 => generated 311 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:01:22.032570 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6f1aa-254b-4fb8-9014-00abda6d0fb0 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:22.100406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:22.151304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6f1aa-254b-4fb8-9014-00abda6d0fb0 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:22.187513 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835196b9-c66a-45b6-9638-0dd203b06485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:22.188134 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835196b9-c66a-45b6-9638-0dd203b06485 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:22.188403 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835196b9-c66a-45b6-9638-0dd203b06485 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:22.188656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835196b9-c66a-45b6-9638-0dd203b06485 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:22.269301 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835196b9-c66a-45b6-9638-0dd203b06485 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:22.270834 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835196b9-c66a-45b6-9638-0dd203b06485 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:22.334227 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:22.387300 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 588/1183] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 15:01:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 15:01:22.393808 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97fb6a-180b-40e4-b23e-e5c6ff718302 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:22.487551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:22.504213 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97fb6a-180b-40e4-b23e-e5c6ff718302 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:22.540701 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d7bb6-1770-4dc3-8b26-53b6b49102d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:22.541386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d7bb6-1770-4dc3-8b26-53b6b49102d3 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:22.541641 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d7bb6-1770-4dc3-8b26-53b6b49102d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:22.541935 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d7bb6-1770-4dc3-8b26-53b6b49102d3 None admin] PATH_INFO: `/v3/projects/c46b45bfc1794cf79472334822a62154` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:22.543143 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d7bb6-1770-4dc3-8b26-53b6b49102d3 None admin] RBAC: Authorizing `identity:get_project(project_id=c46b45bfc1794cf79472334822a62154)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:22.544583 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d7bb6-1770-4dc3-8b26-53b6b49102d3 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:22.546242 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 596/1184] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 15:01:22 2022] GET /identity/v3/projects/c46b45bfc1794cf79472334822a62154 => generated 361 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:01:22.552896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835196b9-c66a-45b6-9638-0dd203b06485 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:22.676830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835196b9-c66a-45b6-9638-0dd203b06485 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:22.724114 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c91fa2-f7c9-4b37-a8b4-32677aa22a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:22.724760 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c91fa2-f7c9-4b37-a8b4-32677aa22a3a None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:22.725035 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c91fa2-f7c9-4b37-a8b4-32677aa22a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:22.725324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c91fa2-f7c9-4b37-a8b4-32677aa22a3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:22.726066 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c91fa2-f7c9-4b37-a8b4-32677aa22a3a None admin] RBAC: Adding query filter params (role.id=c426875a15834aa5b122e2a62267d15a, scope.project.id=c46b45bfc1794cf79472334822a62154, user.id=47d28b15d8b3475a812012d92446a7e1) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:01:22.726374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c91fa2-f7c9-4b37-a8b4-32677aa22a3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:22.727939 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c91fa2-f7c9-4b37-a8b4-32677aa22a3a None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:22.734563 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 589/1185] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 15:01:22 2022] GET /identity/v3/role_assignments?user.id=47d28b15d8b3475a812012d92446a7e1&scope.project.id=c46b45bfc1794cf79472334822a62154&role.id=c426875a15834aa5b122e2a62267d15a => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:01:24.893644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97a669-c716-4549-8373-739ce3c64f54 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:24.894247 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97a669-c716-4549-8373-739ce3c64f54 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:24.894247 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97a669-c716-4549-8373-739ce3c64f54 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:24.895509 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 597/1186] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:24.901394 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9f624-601a-4310-8d53-432fc74d3f4a None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:24.901912 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9f624-601a-4310-8d53-432fc74d3f4a None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:24.901912 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9f624-601a-4310-8d53-432fc74d3f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:24.925844 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-c8d9f624-601a-4310-8d53-432fc74d3f4a None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:24.927738 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c8d9f624-601a-4310-8d53-432fc74d3f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:24.943433 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:25.000760 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 590/1187] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:01:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:01:25.264484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97a669-c716-4549-8373-739ce3c64f54 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:25.377609 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97a669-c716-4549-8373-739ce3c64f54 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:25.417676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451bac8a-5dea-4678-a77f-2affdb77b0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:25.418324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bac8a-5dea-4678-a77f-2affdb77b0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:25.418604 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bac8a-5dea-4678-a77f-2affdb77b0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:25.418885 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bac8a-5dea-4678-a77f-2affdb77b0e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:25.430590 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451bac8a-5dea-4678-a77f-2affdb77b0e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 15:01:25.435436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-451bac8a-5dea-4678-a77f-2affdb77b0e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 15:01:25.435436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:01:25.435436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:01:25.435436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:01:25.435436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:01:25.435436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:01:25.435436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:01:25.435436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:01:25.435436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:01:25.435436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:01:25.435436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:01:25.435436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:01:25.435436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 15:01:25.435436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 15:01:25.435436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 15:01:25.435436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 15:01:25.436848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 15:01:25.436848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 15:01:25.436848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 15:01:25.436848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 15:01:25.436848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:01:25.436848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:01:25.436848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:01:25.436848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:01:25.436848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:01:25.436848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:01:25.436848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:01:25.436848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:01:25.436848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:01:25.436848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:01:25.436848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:01:25.436848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:01:25.438503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:01:25.438503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:01:25.438503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:01:25.438503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:01:25.438503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 15:01:25.438503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 15:01:25.438503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 15:01:25.438503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 15:01:25.438503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 15:01:25.438503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 15:01:25.438503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 15:01:25.438503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:01:25.438503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 598/1188] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 15:01:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:01:25.443134 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d9f624-601a-4310-8d53-432fc74d3f4a None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:25.580240 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d9f624-601a-4310-8d53-432fc74d3f4a None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:25.637477 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51517158-132d-4474-b428-2d601b1acd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:25.638400 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51517158-132d-4474-b428-2d601b1acd2a None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:25.638801 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51517158-132d-4474-b428-2d601b1acd2a None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:25.639197 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51517158-132d-4474-b428-2d601b1acd2a None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:25.640233 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51517158-132d-4474-b428-2d601b1acd2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:01:25.640679 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51517158-132d-4474-b428-2d601b1acd2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:25.643068 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51517158-132d-4474-b428-2d601b1acd2a None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:25.658224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 591/1189] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 15:01:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:01:25.665438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451bac8a-5dea-4678-a77f-2affdb77b0e2 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:25.684578 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:25.787095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451bac8a-5dea-4678-a77f-2affdb77b0e2 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:25.825955 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050755e-3758-4877-9c62-43e311198d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:25.826601 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050755e-3758-4877-9c62-43e311198d95 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:25.826872 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050755e-3758-4877-9c62-43e311198d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:25.827145 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050755e-3758-4877-9c62-43e311198d95 None admin] PATH_INFO: `/v3/users/47d28b15d8b3475a812012d92446a7e1` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:25.828318 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050755e-3758-4877-9c62-43e311198d95 None admin] RBAC: Authorizing `identity:get_user(user_id=47d28b15d8b3475a812012d92446a7e1)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:25.829832 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050755e-3758-4877-9c62-43e311198d95 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:25.831502 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 599/1190] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 15:01:25 2022] GET /identity/v3/users/47d28b15d8b3475a812012d92446a7e1 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:01:25.838038 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51517158-132d-4474-b428-2d601b1acd2a None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:25.853112 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:25.947900 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51517158-132d-4474-b428-2d601b1acd2a None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:25.984530 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c128146-85a3-4adf-9ec4-2c17c29b54b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:25.985186 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c128146-85a3-4adf-9ec4-2c17c29b54b6 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:25.985463 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c128146-85a3-4adf-9ec4-2c17c29b54b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:25.985729 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c128146-85a3-4adf-9ec4-2c17c29b54b6 None admin] PATH_INFO: `/v3/projects/c46b45bfc1794cf79472334822a62154` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:25.986858 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c128146-85a3-4adf-9ec4-2c17c29b54b6 None admin] RBAC: Authorizing `identity:get_project(project_id=c46b45bfc1794cf79472334822a62154)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:25.988311 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c128146-85a3-4adf-9ec4-2c17c29b54b6 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:25.989988 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 592/1191] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 15:01:25 2022] GET /identity/v3/projects/c46b45bfc1794cf79472334822a62154 => generated 361 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:01:25.996278 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050755e-3758-4877-9c62-43e311198d95 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:26.086128 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:26.105887 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050755e-3758-4877-9c62-43e311198d95 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:26.145365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f1927-f943-42f5-82ac-c3a61b095fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:26.146035 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f1927-f943-42f5-82ac-c3a61b095fac None admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:26.146321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f1927-f943-42f5-82ac-c3a61b095fac None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:26.146597 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f1927-f943-42f5-82ac-c3a61b095fac None admin] PATH_INFO: `/v3/projects/c46b45bfc1794cf79472334822a62154/users/47d28b15d8b3475a812012d92446a7e1/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:26.148599 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f1927-f943-42f5-82ac-c3a61b095fac None admin] RBAC: Authorizing `identity:create_grant(project_id=c46b45bfc1794cf79472334822a62154, user_id=47d28b15d8b3475a812012d92446a7e1, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:26.150167 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f1927-f943-42f5-82ac-c3a61b095fac None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:26.166494 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 600/1192] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 15:01:25 2022] PUT /identity/v3/projects/c46b45bfc1794cf79472334822a62154/users/47d28b15d8b3475a812012d92446a7e1/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:01:28.332008 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de1964-7156-4673-929a-ad9529f55595 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:28.332638 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de1964-7156-4673-929a-ad9529f55595 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:28.332638 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de1964-7156-4673-929a-ad9529f55595 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:28.333944 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 593/1193] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:01:28.343525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ae9a3-943e-46ef-ab42-c02162838b80 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:28.344412 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ae9a3-943e-46ef-ab42-c02162838b80 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:28.344817 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ae9a3-943e-46ef-ab42-c02162838b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:28.372952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-958ae9a3-943e-46ef-ab42-c02162838b80 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:01:28.375038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-958ae9a3-943e-46ef-ab42-c02162838b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:01:28.450946 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 601/1194] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:01:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:01:28.724229 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de1964-7156-4673-929a-ad9529f55595 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:28.806016 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:28.830396 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de1964-7156-4673-929a-ad9529f55595 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:28.864634 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161f4d5-80e1-4316-9d31-c647fd33d01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:28.865294 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161f4d5-80e1-4316-9d31-c647fd33d01a None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:28.865564 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161f4d5-80e1-4316-9d31-c647fd33d01a None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:28.865829 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161f4d5-80e1-4316-9d31-c647fd33d01a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:28.876933 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9161f4d5-80e1-4316-9d31-c647fd33d01a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 15:01:28.881861 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-9161f4d5-80e1-4316-9d31-c647fd33d01a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 15:01:28.881861 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:01:28.881861 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:01:28.881861 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:01:28.881861 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:01:28.881861 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:01:28.881861 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:01:28.881861 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:01:28.881861 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:01:28.881861 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:01:28.881861 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:01:28.881861 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:01:28.881861 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 15:01:28.881861 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 15:01:28.881861 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 15:01:28.881861 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 15:01:28.883223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 15:01:28.883223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 15:01:28.883223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 15:01:28.883223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 15:01:28.883223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:01:28.883223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:01:28.883223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:01:28.883223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:01:28.883223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:01:28.883223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:01:28.883223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:01:28.883223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:01:28.883223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:01:28.883223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:01:28.883223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:01:28.883223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:01:28.884200 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:01:28.884200 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:01:28.884200 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:01:28.884200 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:01:28.884200 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 15:01:28.884200 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 15:01:28.884200 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 15:01:28.884200 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 15:01:28.884200 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 15:01:28.884200 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 15:01:28.884200 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 15:01:28.884200 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:01:28.884200 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 594/1195] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 15:01:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:01:28.890158 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ae9a3-943e-46ef-ab42-c02162838b80 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:28.977561 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:29.014492 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ae9a3-943e-46ef-ab42-c02162838b80 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:29.055616 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82857f7-c177-4fea-b7c6-334f38b5df0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:29.056259 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82857f7-c177-4fea-b7c6-334f38b5df0d None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:29.056532 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82857f7-c177-4fea-b7c6-334f38b5df0d None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:29.056803 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82857f7-c177-4fea-b7c6-334f38b5df0d None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:29.057546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82857f7-c177-4fea-b7c6-334f38b5df0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:01:29.057849 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82857f7-c177-4fea-b7c6-334f38b5df0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:29.059314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82857f7-c177-4fea-b7c6-334f38b5df0d None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:29.069391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 602/1196] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 15:01:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:01:29.076269 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161f4d5-80e1-4316-9d31-c647fd33d01a None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:29.192089 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161f4d5-80e1-4316-9d31-c647fd33d01a None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:29.228247 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4deb03-e49c-4d15-9070-ce1c3b88f377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:29.228802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4deb03-e49c-4d15-9070-ce1c3b88f377 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:29.229079 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4deb03-e49c-4d15-9070-ce1c3b88f377 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:29.229384 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4deb03-e49c-4d15-9070-ce1c3b88f377 None admin] PATH_INFO: `/v3/users/47d28b15d8b3475a812012d92446a7e1` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:29.230576 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4deb03-e49c-4d15-9070-ce1c3b88f377 None admin] RBAC: Authorizing `identity:get_user(user_id=47d28b15d8b3475a812012d92446a7e1)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:29.232065 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4deb03-e49c-4d15-9070-ce1c3b88f377 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:29.233702 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 595/1197] 10.222.0.44 () {56 vars in 1177 bytes} [Sun Sep 11 15:01:29 2022] GET /identity/v3/users/47d28b15d8b3475a812012d92446a7e1 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:01:29.240006 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82857f7-c177-4fea-b7c6-334f38b5df0d None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:29.349575 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82857f7-c177-4fea-b7c6-334f38b5df0d None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:29.390057 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372ae1d-9caa-4eae-89d7-83437c8c0e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:29.390646 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372ae1d-9caa-4eae-89d7-83437c8c0e23 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:29.390946 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372ae1d-9caa-4eae-89d7-83437c8c0e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:29.391213 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372ae1d-9caa-4eae-89d7-83437c8c0e23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:29.394091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:29.463219 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372ae1d-9caa-4eae-89d7-83437c8c0e23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:29.464675 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372ae1d-9caa-4eae-89d7-83437c8c0e23 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:29.567707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 603/1198] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 15:01:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 15:01:29.574860 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4deb03-e49c-4d15-9070-ce1c3b88f377 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:29.584832 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:29.714458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4deb03-e49c-4d15-9070-ce1c3b88f377 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:29.758690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f10cd4-701b-4409-931e-ce38e9bff1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:29.759353 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f10cd4-701b-4409-931e-ce38e9bff1ce None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:29.759631 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f10cd4-701b-4409-931e-ce38e9bff1ce None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:29.759902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f10cd4-701b-4409-931e-ce38e9bff1ce None admin] PATH_INFO: `/v3/projects/c46b45bfc1794cf79472334822a62154` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:29.761219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f10cd4-701b-4409-931e-ce38e9bff1ce None admin] RBAC: Authorizing `identity:get_project(project_id=c46b45bfc1794cf79472334822a62154)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:29.762701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f10cd4-701b-4409-931e-ce38e9bff1ce None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:29.764455 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 596/1199] 10.222.0.44 () {56 vars in 1186 bytes} [Sun Sep 11 15:01:29 2022] GET /identity/v3/projects/c46b45bfc1794cf79472334822a62154 => generated 361 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:01:29.771283 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372ae1d-9caa-4eae-89d7-83437c8c0e23 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:01:29.809359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:29.890355 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372ae1d-9caa-4eae-89d7-83437c8c0e23 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:01:29.930850 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b1e27-d2b8-4671-8577-5da449f85cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:01:29.931477 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b1e27-d2b8-4671-8577-5da449f85cdb None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:01:29.931745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b1e27-d2b8-4671-8577-5da449f85cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:01:29.932007 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b1e27-d2b8-4671-8577-5da449f85cdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:01:29.932744 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b1e27-d2b8-4671-8577-5da449f85cdb None admin] RBAC: Adding query filter params (role.id=c426875a15834aa5b122e2a62267d15a, scope.project.id=c46b45bfc1794cf79472334822a62154, user.id=47d28b15d8b3475a812012d92446a7e1) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:01:29.933043 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b1e27-d2b8-4671-8577-5da449f85cdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:01:29.934523 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b1e27-d2b8-4671-8577-5da449f85cdb None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:01:29.941274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 604/1200] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 15:01:29 2022] GET /identity/v3/role_assignments?user.id=47d28b15d8b3475a812012d92446a7e1&scope.project.id=c46b45bfc1794cf79472334822a62154&role.id=c426875a15834aa5b122e2a62267d15a => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:01:39.753455 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=63 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:01:39.938254 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=51 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:26.409559 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2237b0-787c-4683-8f4a-89ad127be076 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:26.410190 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2237b0-787c-4683-8f4a-89ad127be076 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:26.410190 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2237b0-787c-4683-8f4a-89ad127be076 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:26.411782 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 597/1201] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:02:26.418983 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b50b68-b26c-4494-91bd-2643d2decdd4 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:26.419647 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b50b68-b26c-4494-91bd-2643d2decdd4 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:26.419647 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b50b68-b26c-4494-91bd-2643d2decdd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:26.447356 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-e5b50b68-b26c-4494-91bd-2643d2decdd4 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:02:26.448975 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-e5b50b68-b26c-4494-91bd-2643d2decdd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:02:26.536696 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 605/1202] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:02:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 15:02:26.817745 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2237b0-787c-4683-8f4a-89ad127be076 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:26.942455 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2237b0-787c-4683-8f4a-89ad127be076 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:26.980507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79cc8fc-0ce0-45f8-8d8f-20b92fd8cac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:26.981243 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79cc8fc-0ce0-45f8-8d8f-20b92fd8cac6 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:26.981514 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79cc8fc-0ce0-45f8-8d8f-20b92fd8cac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:26.981747 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79cc8fc-0ce0-45f8-8d8f-20b92fd8cac6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:26.992749 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79cc8fc-0ce0-45f8-8d8f-20b92fd8cac6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:26.994246 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79cc8fc-0ce0-45f8-8d8f-20b92fd8cac6 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:27.021651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-b79cc8fc-0ce0-45f8-8d8f-20b92fd8cac6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:02:27.021651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:27.021651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:27.021651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:27.021651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:27.021651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:27.021651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:27.021651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:27.021651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:27.021651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:27.021651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:27.021651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:27.021651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 15:02:27.021651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 15:02:27.021651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 15:02:27.021651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 15:02:27.022809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:27.022809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:27.022809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:27.022809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:27.022809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:27.022809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:27.022809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:27.022809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:27.022809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:27.022809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:27.022809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:27.022809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:27.022809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:27.022809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:27.022809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:27.022809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:27.022809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 15:02:27.023795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 15:02:27.023795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:02:27.023795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:02:27.023795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 598/1203] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:02:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:02:27.026079 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b50b68-b26c-4494-91bd-2643d2decdd4 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:27.150131 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b50b68-b26c-4494-91bd-2643d2decdd4 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:27.188638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:27.196553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091f36e-0c4d-4610-8496-57560712e287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:27.197278 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091f36e-0c4d-4610-8496-57560712e287 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:27.197633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091f36e-0c4d-4610-8496-57560712e287 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:27.197914 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091f36e-0c4d-4610-8496-57560712e287 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:27.198666 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e091f36e-0c4d-4610-8496-57560712e287 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:27.199004 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e091f36e-0c4d-4610-8496-57560712e287 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:27.200518 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e091f36e-0c4d-4610-8496-57560712e287 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:27.213314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 606/1204] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 15:02:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:02:27.220080 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79cc8fc-0ce0-45f8-8d8f-20b92fd8cac6 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:27.301327 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:27.354440 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79cc8fc-0ce0-45f8-8d8f-20b92fd8cac6 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:27.399871 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8fc44-6d44-4fe7-830a-da328af17012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:27.400996 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8fc44-6d44-4fe7-830a-da328af17012 None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:27.401218 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8fc44-6d44-4fe7-830a-da328af17012 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:27.401494 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8fc44-6d44-4fe7-830a-da328af17012 None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:27.402158 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a8fc44-6d44-4fe7-830a-da328af17012 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:27.403638 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a8fc44-6d44-4fe7-830a-da328af17012 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:27.432586 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 599/1205] 10.222.0.44 () {60 vars in 1132 bytes} [Sun Sep 11 15:02:27 2022] POST /identity/v3/users => generated 251 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:02:30.082273 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef2ca8-d682-4b76-8e54-c9fdd18bd339 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:30.082896 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef2ca8-d682-4b76-8e54-c9fdd18bd339 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:30.082896 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef2ca8-d682-4b76-8e54-c9fdd18bd339 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:30.084145 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 607/1206] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:02:30.090947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bca58a-46f8-469f-8ccf-1c0300e500ff None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:30.093473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bca58a-46f8-469f-8ccf-1c0300e500ff None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:30.093473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bca58a-46f8-469f-8ccf-1c0300e500ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:30.113780 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-d5bca58a-46f8-469f-8ccf-1c0300e500ff None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:02:30.114963 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-d5bca58a-46f8-469f-8ccf-1c0300e500ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:02:30.202281 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 600/1207] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:02:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:02:30.482773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef2ca8-d682-4b76-8e54-c9fdd18bd339 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:30.603924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef2ca8-d682-4b76-8e54-c9fdd18bd339 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:30.646002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41653c83-96ac-4755-88a4-5a02b7b3678d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:30.646792 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41653c83-96ac-4755-88a4-5a02b7b3678d None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:30.647083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41653c83-96ac-4755-88a4-5a02b7b3678d None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:30.647451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41653c83-96ac-4755-88a4-5a02b7b3678d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:30.661436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41653c83-96ac-4755-88a4-5a02b7b3678d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 15:02:30.667190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-41653c83-96ac-4755-88a4-5a02b7b3678d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 15:02:30.667190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:30.667190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:30.667190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:30.667190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:30.667190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:30.667190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:30.667190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:30.667190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:30.667190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:30.667190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:30.667190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:30.667190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 15:02:30.667190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 15:02:30.667190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 15:02:30.667190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 15:02:30.668364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 15:02:30.668364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 15:02:30.668364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 15:02:30.668364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 15:02:30.668364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:30.668364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:30.668364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:30.668364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:30.668364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:30.668364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:30.668364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:30.668364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:30.668364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:30.668364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:30.668364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:30.668364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:30.669809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:30.669809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:30.669809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:30.669809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:30.669809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 15:02:30.669809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 15:02:30.669809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 15:02:30.669809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 15:02:30.669809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 15:02:30.669809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 15:02:30.669809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 15:02:30.669809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:02:30.669809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 608/1208] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 15:02:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:02:30.675274 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bca58a-46f8-469f-8ccf-1c0300e500ff None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:30.726385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:30.815989 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bca58a-46f8-469f-8ccf-1c0300e500ff None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:30.871724 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2271e-86ea-4085-b640-ffe478f358fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:30.872468 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2271e-86ea-4085-b640-ffe478f358fa None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:30.872786 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2271e-86ea-4085-b640-ffe478f358fa None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:30.873249 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2271e-86ea-4085-b640-ffe478f358fa None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:30.874347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e2271e-86ea-4085-b640-ffe478f358fa None admin] RBAC: Adding query filter params (name=service) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:30.874580 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e2271e-86ea-4085-b640-ffe478f358fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:30.876116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e2271e-86ea-4085-b640-ffe478f358fa None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:30.887236 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 601/1209] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 15:02:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:02:30.894441 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41653c83-96ac-4755-88a4-5a02b7b3678d None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:30.963292 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:31.027571 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41653c83-96ac-4755-88a4-5a02b7b3678d None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:31.072526 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84ac9b-8531-4abc-98f8-313eed43bae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:31.073159 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84ac9b-8531-4abc-98f8-313eed43bae3 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:31.074725 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84ac9b-8531-4abc-98f8-313eed43bae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:31.074725 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84ac9b-8531-4abc-98f8-313eed43bae3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:31.086942 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e84ac9b-8531-4abc-98f8-313eed43bae3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:31.088567 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e84ac9b-8531-4abc-98f8-313eed43bae3 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:31.108033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-6e84ac9b-8531-4abc-98f8-313eed43bae3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:02:31.108033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:31.108033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:31.108033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:31.108033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:31.108033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:31.108033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:31.108033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:31.108033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:31.108033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:31.108033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:31.108033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:31.108033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 15:02:31.108033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 15:02:31.108033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 15:02:31.108033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 15:02:31.113451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:31.113451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:31.113451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:31.113451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:31.113451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:31.113451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:31.113451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:31.113451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:31.113451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:31.113451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:31.113451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:31.113451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:31.113451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:31.113451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:31.113451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:31.113451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:31.113451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 15:02:31.114723 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 15:02:31.114723 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:02:31.114723 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:02:31.114723 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 609/1210] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:02:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:02:31.115460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2271e-86ea-4085-b640-ffe478f358fa None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:31.274537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:31.283198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2271e-86ea-4085-b640-ffe478f358fa None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:31.334491 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75dff23-371b-412c-b7d3-05a4a47bf96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:31.335298 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75dff23-371b-412c-b7d3-05a4a47bf96d None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:31.335691 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75dff23-371b-412c-b7d3-05a4a47bf96d None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:31.335989 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75dff23-371b-412c-b7d3-05a4a47bf96d None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:31.336737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75dff23-371b-412c-b7d3-05a4a47bf96d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:31.337027 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75dff23-371b-412c-b7d3-05a4a47bf96d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:31.338563 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75dff23-371b-412c-b7d3-05a4a47bf96d None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:31.352253 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 602/1211] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 15:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:02:31.360813 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84ac9b-8531-4abc-98f8-313eed43bae3 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:31.492372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84ac9b-8531-4abc-98f8-313eed43bae3 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:31.493947 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:31.534448 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce86216-133d-42a7-88ae-6ce4e7fd3a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:31.535374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce86216-133d-42a7-88ae-6ce4e7fd3a77 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:31.535642 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce86216-133d-42a7-88ae-6ce4e7fd3a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:31.535944 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce86216-133d-42a7-88ae-6ce4e7fd3a77 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:31.546080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce86216-133d-42a7-88ae-6ce4e7fd3a77 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:31.547684 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce86216-133d-42a7-88ae-6ce4e7fd3a77 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:31.561958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-cce86216-133d-42a7-88ae-6ce4e7fd3a77 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 15:02:31.561958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:31.561958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:31.561958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:31.561958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:31.561958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:31.561958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:31.561958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:31.561958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:31.561958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:31.561958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:31.561958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:31.561958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 15:02:31.561958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 15:02:31.561958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 15:02:31.561958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 15:02:31.563056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:31.563056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:31.563056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 15:02:31.563056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 15:02:31.563056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 15:02:31.563056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 15:02:31.563056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:31.563056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:31.563056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:31.563056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:31.563056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:31.563056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:31.563056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:31.563056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:31.563056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:31.563056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:31.563056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:31.563056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:31.564088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:31.564088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:31.564088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 15:02:31.564088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 15:02:31.564088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 15:02:31.564088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 15:02:31.564088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 15:02:31.564088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 15:02:31.564088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 15:02:31.564088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:02:31.564088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 610/1212] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:02:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:02:31.569833 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75dff23-371b-412c-b7d3-05a4a47bf96d None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:31.690000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75dff23-371b-412c-b7d3-05a4a47bf96d None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:31.730194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c1f73-e06d-4b84-a161-7dc9fdb361be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:31.730809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c1f73-e06d-4b84-a161-7dc9fdb361be None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:31.731082 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c1f73-e06d-4b84-a161-7dc9fdb361be None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:31.731348 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c1f73-e06d-4b84-a161-7dc9fdb361be None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:31.732086 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c1f73-e06d-4b84-a161-7dc9fdb361be None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:31.732388 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c1f73-e06d-4b84-a161-7dc9fdb361be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:31.733860 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c1f73-e06d-4b84-a161-7dc9fdb361be None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:31.752867 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 603/1213] 10.222.0.44 () {56 vars in 1143 bytes} [Sun Sep 11 15:02:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:02:31.758731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce86216-133d-42a7-88ae-6ce4e7fd3a77 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:31.887968 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce86216-133d-42a7-88ae-6ce4e7fd3a77 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:31.925587 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd662d-0d17-46f8-9fa0-4c28d58b480e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:31.926205 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd662d-0d17-46f8-9fa0-4c28d58b480e None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:31.926478 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd662d-0d17-46f8-9fa0-4c28d58b480e None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:31.926748 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd662d-0d17-46f8-9fa0-4c28d58b480e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:31.970193 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:32.011902 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd662d-0d17-46f8-9fa0-4c28d58b480e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:32.013595 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd662d-0d17-46f8-9fa0-4c28d58b480e None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:32.137961 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 611/1214] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 15:02:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 15:02:32.143809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c1f73-e06d-4b84-a161-7dc9fdb361be None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:32.166762 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:32.275255 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c1f73-e06d-4b84-a161-7dc9fdb361be None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:32.319299 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0169b1e2-428f-473b-94a2-e0e808e47a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:32.319983 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169b1e2-428f-473b-94a2-e0e808e47a33 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:32.320254 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169b1e2-428f-473b-94a2-e0e808e47a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:32.320519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169b1e2-428f-473b-94a2-e0e808e47a33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:32.331430 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0169b1e2-428f-473b-94a2-e0e808e47a33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:32.333019 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0169b1e2-428f-473b-94a2-e0e808e47a33 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:32.348588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-0169b1e2-428f-473b-94a2-e0e808e47a33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:02:32.348588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:32.348588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:32.348588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:32.348588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:32.348588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:32.348588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:32.348588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:32.348588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:32.348588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:32.348588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:32.348588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:32.348588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 15:02:32.348588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 15:02:32.348588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 15:02:32.348588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 15:02:32.350039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:32.350039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:32.350039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:32.350039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:32.350039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:32.350039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:32.350039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:32.350039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:32.350039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:32.350039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:32.350039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:32.350039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:32.350039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:32.350039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:32.350039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:32.350039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:32.350039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 15:02:32.351310 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 15:02:32.351310 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:02:32.351310 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:02:32.353300 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 604/1215] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:02:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:02:32.356882 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd662d-0d17-46f8-9fa0-4c28d58b480e None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:32.421106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:32.461347 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd662d-0d17-46f8-9fa0-4c28d58b480e None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:32.497349 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee4370-cbaa-4abc-bc74-55c0e02f2542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:32.498211 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee4370-cbaa-4abc-bc74-55c0e02f2542 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:32.498482 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee4370-cbaa-4abc-bc74-55c0e02f2542 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:32.498783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee4370-cbaa-4abc-bc74-55c0e02f2542 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:32.499478 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee4370-cbaa-4abc-bc74-55c0e02f2542 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:32.499731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee4370-cbaa-4abc-bc74-55c0e02f2542 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:32.501108 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee4370-cbaa-4abc-bc74-55c0e02f2542 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:32.513084 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 612/1216] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 15:02:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:02:32.518894 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0169b1e2-428f-473b-94a2-e0e808e47a33 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:32.600088 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:32.639708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0169b1e2-428f-473b-94a2-e0e808e47a33 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:32.679775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc9e39-8d5c-47c6-9dde-f43ac481daae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:32.680415 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc9e39-8d5c-47c6-9dde-f43ac481daae None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:32.680688 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc9e39-8d5c-47c6-9dde-f43ac481daae None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:32.680952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc9e39-8d5c-47c6-9dde-f43ac481daae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:32.689810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc9e39-8d5c-47c6-9dde-f43ac481daae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:32.691706 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc9e39-8d5c-47c6-9dde-f43ac481daae None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:32.705917 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-0dfc9e39-8d5c-47c6-9dde-f43ac481daae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 15:02:32.705917 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:32.705917 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:32.705917 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:32.705917 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:32.705917 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:32.705917 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:32.705917 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:32.705917 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:32.705917 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:32.705917 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:32.705917 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:32.705917 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 15:02:32.705917 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 15:02:32.705917 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 15:02:32.709508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 15:02:32.709508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:32.709508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:32.709508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:32.709508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:32.709508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:32.709508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:32.709508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:32.709508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:32.709508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:32.709508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:32.709508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:32.709508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:32.709508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:32.709508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:32.709508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:32.709508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:32.710598 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 15:02:32.710598 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 15:02:32.710598 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 15:02:32.710598 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 15:02:32.710598 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 15:02:32.710598 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 15:02:32.710598 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 15:02:32.710598 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:02:32.710598 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 605/1217] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 15:02:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:02:32.713958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee4370-cbaa-4abc-bc74-55c0e02f2542 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:32.832387 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee4370-cbaa-4abc-bc74-55c0e02f2542 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:32.859889 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:32.878664 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9260c5-189a-44f9-aff8-e83401120175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:32.879494 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9260c5-189a-44f9-aff8-e83401120175 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:32.879866 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9260c5-189a-44f9-aff8-e83401120175 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:32.880225 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9260c5-189a-44f9-aff8-e83401120175 None admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:32.881313 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9260c5-189a-44f9-aff8-e83401120175 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:32.881772 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9260c5-189a-44f9-aff8-e83401120175 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:32.883418 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9260c5-189a-44f9-aff8-e83401120175 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:32.896498 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 613/1218] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 15:02:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:02:32.904333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc9e39-8d5c-47c6-9dde-f43ac481daae None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:33.017399 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc9e39-8d5c-47c6-9dde-f43ac481daae None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:33.048815 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:33.054887 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6ab75-e218-43a0-ae1f-257856c1d61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:33.055529 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6ab75-e218-43a0-ae1f-257856c1d61f None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:33.056062 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6ab75-e218-43a0-ae1f-257856c1d61f None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:33.056343 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6ab75-e218-43a0-ae1f-257856c1d61f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:33.057115 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab6ab75-e218-43a0-ae1f-257856c1d61f None admin] RBAC: Adding query filter params (role.id=ea3b2f1eb283418b8823cb4fff80edba, scope.project.id=70ac515938a24c45ae6302dbac46a1ab, user.id=ead06362f8a2444bbfe1405cd602f1c8) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:33.057444 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab6ab75-e218-43a0-ae1f-257856c1d61f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:33.058911 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab6ab75-e218-43a0-ae1f-257856c1d61f None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:33.065626 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 606/1219] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 15:02:32 2022] GET /identity/v3/role_assignments?user.id=ead06362f8a2444bbfe1405cd602f1c8&scope.project.id=70ac515938a24c45ae6302dbac46a1ab&role.id=ea3b2f1eb283418b8823cb4fff80edba => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:02:35.290035 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db38378-c9e6-42d7-80ff-a2da80aea286 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:35.290612 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db38378-c9e6-42d7-80ff-a2da80aea286 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:35.290612 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db38378-c9e6-42d7-80ff-a2da80aea286 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:35.291924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 614/1220] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:02:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:02:35.297696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e22b1c-12bf-4935-9481-e3685cedb411 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:35.298161 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e22b1c-12bf-4935-9481-e3685cedb411 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:35.298324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e22b1c-12bf-4935-9481-e3685cedb411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:35.319838 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-b3e22b1c-12bf-4935-9481-e3685cedb411 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:02:35.321125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b3e22b1c-12bf-4935-9481-e3685cedb411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:02:35.392786 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 607/1221] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:02:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:02:35.670650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db38378-c9e6-42d7-80ff-a2da80aea286 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:35.785946 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db38378-c9e6-42d7-80ff-a2da80aea286 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:35.822658 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de845e25-2c57-477e-81e9-c8027c44204b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:35.823302 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de845e25-2c57-477e-81e9-c8027c44204b None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:35.823574 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de845e25-2c57-477e-81e9-c8027c44204b None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:35.823841 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de845e25-2c57-477e-81e9-c8027c44204b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:35.835077 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de845e25-2c57-477e-81e9-c8027c44204b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 15:02:35.840359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-de845e25-2c57-477e-81e9-c8027c44204b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 15:02:35.840359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:35.840359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:35.840359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:35.840359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:35.840359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:35.840359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:35.840359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:35.840359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:35.840359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:35.840359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:35.840359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:35.840359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 15:02:35.840359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 15:02:35.840359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 15:02:35.840359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 15:02:35.841525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 15:02:35.841525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 15:02:35.841525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 15:02:35.841525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 15:02:35.841525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:35.841525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:35.841525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:35.841525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:35.841525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:35.841525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:35.841525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:35.841525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:35.841525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:35.841525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:35.841525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:35.841525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:35.842763 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:35.842763 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:35.842763 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:35.842763 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:35.842763 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 15:02:35.842763 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 15:02:35.842763 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 15:02:35.842763 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 15:02:35.842763 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 15:02:35.842763 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 15:02:35.842763 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 15:02:35.842763 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:02:35.842763 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 615/1222] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 15:02:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 15:02:35.848687 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e22b1c-12bf-4935-9481-e3685cedb411 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:35.957383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e22b1c-12bf-4935-9481-e3685cedb411 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:35.984210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:35.996884 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007817e-965a-4ef2-b137-e59a85595f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:35.997541 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007817e-965a-4ef2-b137-e59a85595f1b None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:35.997815 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007817e-965a-4ef2-b137-e59a85595f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:35.998078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007817e-965a-4ef2-b137-e59a85595f1b None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:35.998760 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b007817e-965a-4ef2-b137-e59a85595f1b None admin] RBAC: Adding query filter params (name=service) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:35.999032 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b007817e-965a-4ef2-b137-e59a85595f1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:36.000395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b007817e-965a-4ef2-b137-e59a85595f1b None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:36.008845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 608/1223] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 15:02:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:02:36.014990 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de845e25-2c57-477e-81e9-c8027c44204b None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:36.062421 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:36.130564 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de845e25-2c57-477e-81e9-c8027c44204b None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:36.172201 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4132c6-fafc-41b2-aba8-fe54b884fcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:36.172836 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4132c6-fafc-41b2-aba8-fe54b884fcab None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:36.173085 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4132c6-fafc-41b2-aba8-fe54b884fcab None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:36.173410 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4132c6-fafc-41b2-aba8-fe54b884fcab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:36.190772 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4132c6-fafc-41b2-aba8-fe54b884fcab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:36.193591 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4132c6-fafc-41b2-aba8-fe54b884fcab None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:36.211185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-8b4132c6-fafc-41b2-aba8-fe54b884fcab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:02:36.211185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:36.211185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:36.211185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:36.211185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:36.211185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:36.211185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:36.211185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:36.211185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:36.211185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:36.211185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:36.211185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:36.211185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 15:02:36.211185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 15:02:36.211185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 15:02:36.211185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 15:02:36.212210 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:36.212210 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:36.212210 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:36.212210 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:36.212210 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:36.212210 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:36.212210 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:36.212210 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:36.212210 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:36.212210 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:36.212210 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:36.212210 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:36.212210 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:36.212210 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:36.212210 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:36.212210 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:36.212210 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 15:02:36.213098 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 15:02:36.213098 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:02:36.213098 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:02:36.213098 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 616/1224] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:02:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:02:36.218850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007817e-965a-4ef2-b137-e59a85595f1b None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:36.340559 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007817e-965a-4ef2-b137-e59a85595f1b None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:36.380941 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae437624-f467-4da1-9840-989c7637adf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:36.381629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae437624-f467-4da1-9840-989c7637adf6 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:36.382091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae437624-f467-4da1-9840-989c7637adf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:36.382368 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae437624-f467-4da1-9840-989c7637adf6 None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:36.383045 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae437624-f467-4da1-9840-989c7637adf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:36.383479 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae437624-f467-4da1-9840-989c7637adf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:36.385283 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae437624-f467-4da1-9840-989c7637adf6 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:36.399589 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 609/1225] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 15:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:02:36.405646 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4132c6-fafc-41b2-aba8-fe54b884fcab None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:36.509049 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:36.516887 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4132c6-fafc-41b2-aba8-fe54b884fcab None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:36.554327 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946cc88-f819-45c1-b03b-8c1523477f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:36.554990 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946cc88-f819-45c1-b03b-8c1523477f95 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:36.555239 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946cc88-f819-45c1-b03b-8c1523477f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:36.555504 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946cc88-f819-45c1-b03b-8c1523477f95 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:36.564467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2946cc88-f819-45c1-b03b-8c1523477f95 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:36.566063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2946cc88-f819-45c1-b03b-8c1523477f95 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:36.578501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-2946cc88-f819-45c1-b03b-8c1523477f95 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 15:02:36.578501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:36.578501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:36.578501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:36.578501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:36.578501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:36.578501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:36.578501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:36.578501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:36.578501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:36.578501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:36.578501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:36.578501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 15:02:36.578501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 15:02:36.578501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 15:02:36.578501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 15:02:36.579815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:36.579815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:36.579815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 15:02:36.579815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 15:02:36.579815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 15:02:36.579815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 15:02:36.579815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:36.579815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:36.579815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:36.579815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:36.579815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:36.579815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:36.579815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:36.579815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:36.579815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:36.579815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:36.579815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:36.579815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:36.580993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:36.580993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:36.580993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 15:02:36.580993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 15:02:36.580993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 15:02:36.580993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 15:02:36.580993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 15:02:36.580993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 15:02:36.580993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 15:02:36.580993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:02:36.584848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 617/1226] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:02:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 15:02:36.586152 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae437624-f467-4da1-9840-989c7637adf6 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:36.641441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:36.717877 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae437624-f467-4da1-9840-989c7637adf6 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:36.764278 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32aafe-7c5f-4656-ade2-36e776913280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:36.764938 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32aafe-7c5f-4656-ade2-36e776913280 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:36.765252 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32aafe-7c5f-4656-ade2-36e776913280 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:36.765594 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32aafe-7c5f-4656-ade2-36e776913280 None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:36.766409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d32aafe-7c5f-4656-ade2-36e776913280 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:36.766725 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d32aafe-7c5f-4656-ade2-36e776913280 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:36.768599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d32aafe-7c5f-4656-ade2-36e776913280 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:36.789875 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 610/1227] 10.222.0.44 () {56 vars in 1143 bytes} [Sun Sep 11 15:02:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:02:36.796428 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946cc88-f819-45c1-b03b-8c1523477f95 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:36.921047 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946cc88-f819-45c1-b03b-8c1523477f95 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:36.962419 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefc6a8-f2af-41f2-8af0-7a6b4a9fae16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:36.963067 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefc6a8-f2af-41f2-8af0-7a6b4a9fae16 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:36.963337 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefc6a8-f2af-41f2-8af0-7a6b4a9fae16 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:36.963602 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefc6a8-f2af-41f2-8af0-7a6b4a9fae16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:36.975785 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaefc6a8-f2af-41f2-8af0-7a6b4a9fae16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:36.977289 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaefc6a8-f2af-41f2-8af0-7a6b4a9fae16 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:36.993368 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-aaefc6a8-f2af-41f2-8af0-7a6b4a9fae16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:02:36.993368 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:36.993368 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:36.993368 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:36.993368 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:36.993368 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:36.993368 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:36.993368 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:36.993368 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:36.993368 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:36.993368 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:36.993368 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:36.993368 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 15:02:36.993368 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 15:02:36.993368 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 15:02:36.993368 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 15:02:36.995259 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:36.995259 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:36.995259 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:36.995259 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:36.995259 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:36.995259 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:36.995259 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:36.995259 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:36.995259 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:36.995259 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:36.995259 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:36.995259 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:36.995259 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:36.995259 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:36.995259 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:36.995259 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:36.995259 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 15:02:36.996926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 15:02:36.996926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:02:36.996926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:02:36.996926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 618/1228] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:02:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:02:37.000730 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32aafe-7c5f-4656-ade2-36e776913280 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:37.113907 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32aafe-7c5f-4656-ade2-36e776913280 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:37.116424 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:37.154931 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7468b1-551b-4a9b-a5a2-6963c7326672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:37.155539 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7468b1-551b-4a9b-a5a2-6963c7326672 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:37.155811 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7468b1-551b-4a9b-a5a2-6963c7326672 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:37.156076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7468b1-551b-4a9b-a5a2-6963c7326672 None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:37.156717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7468b1-551b-4a9b-a5a2-6963c7326672 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:37.157013 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7468b1-551b-4a9b-a5a2-6963c7326672 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:37.158424 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7468b1-551b-4a9b-a5a2-6963c7326672 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:37.169956 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 611/1229] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 15:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:02:37.175945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefc6a8-f2af-41f2-8af0-7a6b4a9fae16 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:37.187367 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:37.309052 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefc6a8-f2af-41f2-8af0-7a6b4a9fae16 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:37.350138 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd61bc-8ea8-4428-abb3-3c87998f400e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:37.350825 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd61bc-8ea8-4428-abb3-3c87998f400e None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:37.351149 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd61bc-8ea8-4428-abb3-3c87998f400e None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:37.351484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd61bc-8ea8-4428-abb3-3c87998f400e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:37.360403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fd61bc-8ea8-4428-abb3-3c87998f400e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:37.362095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fd61bc-8ea8-4428-abb3-3c87998f400e None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:37.375498 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-37fd61bc-8ea8-4428-abb3-3c87998f400e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 15:02:37.375498 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:37.375498 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:37.375498 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:37.375498 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:37.375498 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:37.375498 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:37.375498 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:37.375498 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:37.375498 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:37.375498 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:37.375498 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:37.375498 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 15:02:37.375498 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 15:02:37.375498 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 15:02:37.376621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 15:02:37.376621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:37.376621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:37.376621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:37.376621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:37.376621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:37.376621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:37.376621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:37.376621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:37.376621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:37.376621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:37.376621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:37.376621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:37.376621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:37.376621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:37.376621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:37.376621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:37.377596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 15:02:37.377596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 15:02:37.377596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 15:02:37.377596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 15:02:37.377596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 15:02:37.377596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 15:02:37.377596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 15:02:37.377596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:02:37.377596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 619/1230] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 15:02:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:02:37.383554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7468b1-551b-4a9b-a5a2-6963c7326672 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:37.508927 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7468b1-551b-4a9b-a5a2-6963c7326672 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:37.551122 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a906b93-c6d8-4260-84e0-adc5fdc9dafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:37.551747 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a906b93-c6d8-4260-84e0-adc5fdc9dafb None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:37.552024 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a906b93-c6d8-4260-84e0-adc5fdc9dafb None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:37.552296 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a906b93-c6d8-4260-84e0-adc5fdc9dafb None admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:37.553006 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a906b93-c6d8-4260-84e0-adc5fdc9dafb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:37.553395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a906b93-c6d8-4260-84e0-adc5fdc9dafb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:37.554880 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a906b93-c6d8-4260-84e0-adc5fdc9dafb None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:37.568535 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 612/1231] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 15:02:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:02:37.576956 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd61bc-8ea8-4428-abb3-3c87998f400e None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:37.653158 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:37.698278 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd61bc-8ea8-4428-abb3-3c87998f400e None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:37.741692 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23e535-ce55-4da2-94c3-258fd85cf53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:37.742363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23e535-ce55-4da2-94c3-258fd85cf53e None admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:37.742642 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23e535-ce55-4da2-94c3-258fd85cf53e None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:37.743122 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23e535-ce55-4da2-94c3-258fd85cf53e None admin] PATH_INFO: `/v3/projects/70ac515938a24c45ae6302dbac46a1ab/users/ead06362f8a2444bbfe1405cd602f1c8/roles/ea3b2f1eb283418b8823cb4fff80edba` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:37.760004 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23e535-ce55-4da2-94c3-258fd85cf53e None admin] RBAC: Authorizing `identity:create_grant(project_id=70ac515938a24c45ae6302dbac46a1ab, user_id=ead06362f8a2444bbfe1405cd602f1c8, role_id=ea3b2f1eb283418b8823cb4fff80edba)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:37.761496 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23e535-ce55-4da2-94c3-258fd85cf53e None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:37.776069 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 620/1232] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 15:02:37 2022] PUT /identity/v3/projects/70ac515938a24c45ae6302dbac46a1ab/users/ead06362f8a2444bbfe1405cd602f1c8/roles/ea3b2f1eb283418b8823cb4fff80edba => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:02:39.996476 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43861e45-5a4f-418a-a5b0-3861f76fac7b None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:39.997063 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43861e45-5a4f-418a-a5b0-3861f76fac7b None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:39.997239 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43861e45-5a4f-418a-a5b0-3861f76fac7b None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:39.998898 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 613/1233] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:02:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:02:40.004848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e3e42-cb96-48d5-b801-5f8441bc0e98 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:40.005364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e3e42-cb96-48d5-b801-5f8441bc0e98 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:40.005501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e3e42-cb96-48d5-b801-5f8441bc0e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:40.025352 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-ef9e3e42-cb96-48d5-b801-5f8441bc0e98 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:02:40.026706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-ef9e3e42-cb96-48d5-b801-5f8441bc0e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:02:40.114136 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 621/1234] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:02:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:02:40.420492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43861e45-5a4f-418a-a5b0-3861f76fac7b None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:40.461006 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:40.592813 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43861e45-5a4f-418a-a5b0-3861f76fac7b None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:40.645095 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f9d9e-7021-4f3f-bd07-28af284e9b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:40.646044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f9d9e-7021-4f3f-bd07-28af284e9b12 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:40.646044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f9d9e-7021-4f3f-bd07-28af284e9b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:40.646655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f9d9e-7021-4f3f-bd07-28af284e9b12 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:40.658388 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f9d9e-7021-4f3f-bd07-28af284e9b12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 15:02:40.663202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-b91f9d9e-7021-4f3f-bd07-28af284e9b12 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 15:02:40.663202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:40.663202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:40.663202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:40.663202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:40.663202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:40.663202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:40.663202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:40.663202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:40.663202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:40.663202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:40.663202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:40.663202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 15:02:40.663202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 15:02:40.663202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 15:02:40.663202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 15:02:40.664456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 15:02:40.664456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 15:02:40.664456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 15:02:40.664456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 15:02:40.664456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:40.664456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:40.664456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:40.664456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:40.664456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:40.664456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:40.664456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:40.664456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:40.664456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:40.664456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:40.664456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:40.664456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:40.665404 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:40.665404 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:40.665404 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:40.665404 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:40.665404 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 15:02:40.665404 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 15:02:40.665404 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 15:02:40.665404 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 15:02:40.665404 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 15:02:40.665404 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 15:02:40.665404 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 15:02:40.665404 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:02:40.665404 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 614/1235] 10.222.0.44 () {56 vars in 1102 bytes} [Sun Sep 11 15:02:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:02:40.673006 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e3e42-cb96-48d5-b801-5f8441bc0e98 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:40.725819 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:40.791866 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e3e42-cb96-48d5-b801-5f8441bc0e98 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:40.831189 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37f0d9-8b14-4c55-bfa5-8b22498f3a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:40.831813 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37f0d9-8b14-4c55-bfa5-8b22498f3a11 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:40.832102 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37f0d9-8b14-4c55-bfa5-8b22498f3a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:40.832384 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37f0d9-8b14-4c55-bfa5-8b22498f3a11 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:40.833052 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37f0d9-8b14-4c55-bfa5-8b22498f3a11 None admin] RBAC: Adding query filter params (name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:40.833423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37f0d9-8b14-4c55-bfa5-8b22498f3a11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:40.834859 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37f0d9-8b14-4c55-bfa5-8b22498f3a11 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:40.849206 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 622/1236] 10.222.0.44 () {56 vars in 1103 bytes} [Sun Sep 11 15:02:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:02:40.855746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f9d9e-7021-4f3f-bd07-28af284e9b12 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:40.978170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:41.004320 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f9d9e-7021-4f3f-bd07-28af284e9b12 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:41.056372 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec535a-cb47-4cd2-afb7-09d46ae04e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:41.057293 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec535a-cb47-4cd2-afb7-09d46ae04e22 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:41.057545 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec535a-cb47-4cd2-afb7-09d46ae04e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:41.057877 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec535a-cb47-4cd2-afb7-09d46ae04e22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:41.071761 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec535a-cb47-4cd2-afb7-09d46ae04e22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:41.073275 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec535a-cb47-4cd2-afb7-09d46ae04e22 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:41.090537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-66ec535a-cb47-4cd2-afb7-09d46ae04e22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:02:41.090537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:41.090537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:41.090537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:41.090537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:41.090537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:41.090537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:41.090537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:41.090537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:41.090537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:41.090537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:41.090537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:41.090537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 15:02:41.090537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 15:02:41.090537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 15:02:41.090537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 15:02:41.092519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:41.092519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:41.092519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:41.092519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:41.092519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:41.092519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:41.092519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:41.092519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:41.092519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:41.092519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:41.092519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:41.092519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:41.092519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:41.092519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:41.092519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:41.092519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:41.092519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 15:02:41.093826 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 15:02:41.093826 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:02:41.093826 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:02:41.093826 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 615/1237] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:02:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:02:41.100065 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37f0d9-8b14-4c55-bfa5-8b22498f3a11 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:41.241345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37f0d9-8b14-4c55-bfa5-8b22498f3a11 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:41.254649 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:41.285238 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b30e66-c387-4da3-9074-80944a28c4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:41.285932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b30e66-c387-4da3-9074-80944a28c4f2 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:41.286254 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b30e66-c387-4da3-9074-80944a28c4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:41.286549 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b30e66-c387-4da3-9074-80944a28c4f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:41.287290 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b30e66-c387-4da3-9074-80944a28c4f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:41.287687 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b30e66-c387-4da3-9074-80944a28c4f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:41.289276 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b30e66-c387-4da3-9074-80944a28c4f2 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:41.304574 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 623/1238] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 15:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:02:41.311783 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec535a-cb47-4cd2-afb7-09d46ae04e22 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:41.440475 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec535a-cb47-4cd2-afb7-09d46ae04e22 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:41.487740 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ebbc6-b832-4244-995d-67fa40ab6e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:41.488463 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ebbc6-b832-4244-995d-67fa40ab6e04 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:41.488463 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ebbc6-b832-4244-995d-67fa40ab6e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:41.489078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ebbc6-b832-4244-995d-67fa40ab6e04 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:41.493155 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:41.501592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482ebbc6-b832-4244-995d-67fa40ab6e04 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:41.504857 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482ebbc6-b832-4244-995d-67fa40ab6e04 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:41.526738 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-482ebbc6-b832-4244-995d-67fa40ab6e04 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 15:02:41.526738 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:41.526738 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:41.526738 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:41.526738 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:41.526738 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:41.526738 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:41.526738 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:41.526738 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:41.526738 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:41.526738 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:41.526738 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:41.526738 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 15:02:41.526738 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 15:02:41.526738 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 15:02:41.526738 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 15:02:41.528709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:41.528709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:41.528709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 15:02:41.528709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 15:02:41.528709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 15:02:41.528709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 15:02:41.528709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:41.528709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:41.528709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:41.528709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:41.528709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:41.528709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:41.528709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:41.528709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:41.528709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:41.528709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:41.528709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:41.528709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:41.530409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:41.530409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:41.530409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 15:02:41.530409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 15:02:41.530409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 15:02:41.530409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 15:02:41.530409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 15:02:41.530409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 15:02:41.530409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 15:02:41.530409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:02:41.530409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 616/1239] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:02:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:02:41.536430 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b30e66-c387-4da3-9074-80944a28c4f2 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:41.680258 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b30e66-c387-4da3-9074-80944a28c4f2 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:41.728886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e790d-49b1-450b-a722-4987b1e6977c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:41.729461 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e790d-49b1-450b-a722-4987b1e6977c None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:41.729786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e790d-49b1-450b-a722-4987b1e6977c None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:41.730005 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e790d-49b1-450b-a722-4987b1e6977c None admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:41.730740 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e790d-49b1-450b-a722-4987b1e6977c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:41.731041 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e790d-49b1-450b-a722-4987b1e6977c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:41.732515 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e790d-49b1-450b-a722-4987b1e6977c None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:41.752213 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 624/1240] 10.222.0.44 () {56 vars in 1143 bytes} [Sun Sep 11 15:02:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:02:41.758380 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ebbc6-b832-4244-995d-67fa40ab6e04 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:41.921853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ebbc6-b832-4244-995d-67fa40ab6e04 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:41.976046 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24acb61b-857f-427f-bb82-f519a46d565f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:41.976717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24acb61b-857f-427f-bb82-f519a46d565f None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:41.977060 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24acb61b-857f-427f-bb82-f519a46d565f None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:41.977436 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24acb61b-857f-427f-bb82-f519a46d565f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:42.056567 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:42.066430 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24acb61b-857f-427f-bb82-f519a46d565f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:42.069394 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24acb61b-857f-427f-bb82-f519a46d565f None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:42.208338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 617/1241] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 15:02:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 15:02:42.215964 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e790d-49b1-450b-a722-4987b1e6977c None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:42.248454 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:42.350015 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e790d-49b1-450b-a722-4987b1e6977c None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:42.393537 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753fe27a-d3e5-40bd-bb3a-2b3a9728b28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:42.394194 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753fe27a-d3e5-40bd-bb3a-2b3a9728b28f None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:42.394466 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753fe27a-d3e5-40bd-bb3a-2b3a9728b28f None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:42.394727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753fe27a-d3e5-40bd-bb3a-2b3a9728b28f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:42.407559 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753fe27a-d3e5-40bd-bb3a-2b3a9728b28f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:42.409482 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753fe27a-d3e5-40bd-bb3a-2b3a9728b28f None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:42.428083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-753fe27a-d3e5-40bd-bb3a-2b3a9728b28f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:02:42.428083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:42.428083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:42.428083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:42.428083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:42.428083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:42.428083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:42.428083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:42.428083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:42.428083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:42.428083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:42.428083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:42.428083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 15:02:42.428083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 15:02:42.428083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 15:02:42.428083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 15:02:42.429706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:42.429706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:42.429706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:42.429706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:42.429706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:42.429706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:42.429706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:42.429706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:42.429706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:42.429706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:42.429706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:42.429706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:42.429706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:42.429706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:42.429706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:42.429706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:42.429706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 15:02:42.431258 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 15:02:42.431258 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:02:42.431258 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:02:42.431258 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 625/1242] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:02:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:02:42.437693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24acb61b-857f-427f-bb82-f519a46d565f None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:42.564220 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:42.571605 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24acb61b-857f-427f-bb82-f519a46d565f None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:42.614019 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99af9b7-10f3-4128-92b6-88cd76674bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:42.614632 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99af9b7-10f3-4128-92b6-88cd76674bab None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:42.614962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99af9b7-10f3-4128-92b6-88cd76674bab None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:42.615287 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99af9b7-10f3-4128-92b6-88cd76674bab None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:42.616086 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99af9b7-10f3-4128-92b6-88cd76674bab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:42.616567 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99af9b7-10f3-4128-92b6-88cd76674bab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:42.619273 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99af9b7-10f3-4128-92b6-88cd76674bab None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:42.639350 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 618/1243] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 15:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:02:42.645734 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753fe27a-d3e5-40bd-bb3a-2b3a9728b28f None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:42.747674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:42.776415 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753fe27a-d3e5-40bd-bb3a-2b3a9728b28f None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:42.824894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c940fba-309d-4806-a4ed-aed06c7870b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:42.825660 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c940fba-309d-4806-a4ed-aed06c7870b9 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:42.825932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c940fba-309d-4806-a4ed-aed06c7870b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:42.826272 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c940fba-309d-4806-a4ed-aed06c7870b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:42.835076 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c940fba-309d-4806-a4ed-aed06c7870b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:42.836856 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c940fba-309d-4806-a4ed-aed06c7870b9 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:42.852206 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-8c940fba-309d-4806-a4ed-aed06c7870b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 15:02:42.852206 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:42.852206 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:42.852206 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:42.852206 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:42.852206 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:42.852206 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:42.852206 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:42.852206 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:42.852206 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:42.852206 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:42.852206 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:42.852206 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 15:02:42.852206 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 15:02:42.852206 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 15:02:42.853831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 15:02:42.853831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:42.853831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:42.853831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:42.853831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:42.853831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:42.853831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:42.853831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:42.853831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:42.853831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:42.853831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:42.853831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:42.853831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:42.853831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:42.853831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:42.853831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:42.853831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:42.855450 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 15:02:42.855450 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 15:02:42.855450 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 15:02:42.855450 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 15:02:42.855450 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 15:02:42.855450 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 15:02:42.855450 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 15:02:42.855450 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:02:42.855450 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 626/1244] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 15:02:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:02:42.861180 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99af9b7-10f3-4128-92b6-88cd76674bab None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:43.007042 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99af9b7-10f3-4128-92b6-88cd76674bab None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:43.050073 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc04bb31-06fb-4c23-b377-9780fc9cf077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:43.050783 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04bb31-06fb-4c23-b377-9780fc9cf077 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:43.051098 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04bb31-06fb-4c23-b377-9780fc9cf077 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:43.051391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04bb31-06fb-4c23-b377-9780fc9cf077 None admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:43.052201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc04bb31-06fb-4c23-b377-9780fc9cf077 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:43.052513 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc04bb31-06fb-4c23-b377-9780fc9cf077 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:43.054232 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc04bb31-06fb-4c23-b377-9780fc9cf077 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:43.068864 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 619/1245] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 15:02:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:02:43.077360 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c940fba-309d-4806-a4ed-aed06c7870b9 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:43.215084 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c940fba-309d-4806-a4ed-aed06c7870b9 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:43.260121 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc73d7-0920-41e1-a1a9-952d87be747b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:43.261046 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc73d7-0920-41e1-a1a9-952d87be747b None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:43.261046 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc73d7-0920-41e1-a1a9-952d87be747b None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:43.261258 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc73d7-0920-41e1-a1a9-952d87be747b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:43.261907 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc73d7-0920-41e1-a1a9-952d87be747b None admin] RBAC: Adding query filter params (role.id=ea3b2f1eb283418b8823cb4fff80edba, scope.project.id=70ac515938a24c45ae6302dbac46a1ab, user.id=ead06362f8a2444bbfe1405cd602f1c8) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:43.262234 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc73d7-0920-41e1-a1a9-952d87be747b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:43.263740 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc73d7-0920-41e1-a1a9-952d87be747b None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:43.271649 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 627/1246] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 15:02:43 2022] GET /identity/v3/role_assignments?user.id=ead06362f8a2444bbfe1405cd602f1c8&scope.project.id=70ac515938a24c45ae6302dbac46a1ab&role.id=ea3b2f1eb283418b8823cb4fff80edba => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:02:45.690547 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc38aa-39c3-4e01-a16d-19002c2950c1 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:45.691101 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc38aa-39c3-4e01-a16d-19002c2950c1 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:45.691101 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc38aa-39c3-4e01-a16d-19002c2950c1 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:45.693791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 620/1247] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:02:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:02:45.699621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d619817-6bc4-41bb-8eeb-b4aa9a370427 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:45.700202 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d619817-6bc4-41bb-8eeb-b4aa9a370427 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:45.700202 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d619817-6bc4-41bb-8eeb-b4aa9a370427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:45.706612 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:45.720821 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-8d619817-6bc4-41bb-8eeb-b4aa9a370427 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:02:45.722057 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-8d619817-6bc4-41bb-8eeb-b4aa9a370427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:02:45.807085 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 628/1248] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:02:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:02:46.114340 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc38aa-39c3-4e01-a16d-19002c2950c1 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:46.135138 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:46.266991 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc38aa-39c3-4e01-a16d-19002c2950c1 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:46.319937 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f7830-4023-4e13-ac6f-888079128ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:46.320591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f7830-4023-4e13-ac6f-888079128ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:46.320872 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f7830-4023-4e13-ac6f-888079128ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:46.321166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f7830-4023-4e13-ac6f-888079128ad6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:46.335477 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f7830-4023-4e13-ac6f-888079128ad6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 15:02:46.340363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-962f7830-4023-4e13-ac6f-888079128ad6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 15:02:46.340363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:46.340363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:46.340363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:46.340363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:46.340363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:46.340363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:46.340363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:46.340363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:46.340363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:46.340363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:46.340363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:46.340363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 15:02:46.340363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 15:02:46.340363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 15:02:46.340363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 15:02:46.342692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 15:02:46.342692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 15:02:46.342692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 15:02:46.342692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 15:02:46.342692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:46.342692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:46.342692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:46.342692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:46.342692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:46.342692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:46.342692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:46.342692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:46.342692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:46.342692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:46.342692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:46.342692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:46.343963 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:46.343963 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:46.343963 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:46.343963 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:46.343963 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 15:02:46.343963 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 15:02:46.343963 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 15:02:46.343963 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 15:02:46.343963 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 15:02:46.343963 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 15:02:46.343963 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 15:02:46.343963 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:02:46.343963 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 621/1249] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 15:02:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 15:02:46.351768 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d619817-6bc4-41bb-8eeb-b4aa9a370427 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:46.488318 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d619817-6bc4-41bb-8eeb-b4aa9a370427 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:46.525859 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:46.531796 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5d6e2-a97d-46c5-a555-b94b279eb363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:46.532417 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5d6e2-a97d-46c5-a555-b94b279eb363 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:46.532695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5d6e2-a97d-46c5-a555-b94b279eb363 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:46.532949 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5d6e2-a97d-46c5-a555-b94b279eb363 None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:46.533651 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5d6e2-a97d-46c5-a555-b94b279eb363 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:46.533950 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5d6e2-a97d-46c5-a555-b94b279eb363 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:46.535300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5d6e2-a97d-46c5-a555-b94b279eb363 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:46.545458 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 629/1250] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 15:02:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:02:46.552041 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f7830-4023-4e13-ac6f-888079128ad6 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:46.664236 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:46.693983 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f7830-4023-4e13-ac6f-888079128ad6 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:46.731336 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cca6fc-d495-401c-ba7f-6ece85eb4fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:46.732036 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cca6fc-d495-401c-ba7f-6ece85eb4fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:46.732310 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cca6fc-d495-401c-ba7f-6ece85eb4fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:46.732576 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cca6fc-d495-401c-ba7f-6ece85eb4fc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:46.744976 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cca6fc-d495-401c-ba7f-6ece85eb4fc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:46.746591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cca6fc-d495-401c-ba7f-6ece85eb4fc5 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:46.766600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-c7cca6fc-d495-401c-ba7f-6ece85eb4fc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:02:46.766600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:46.766600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:46.766600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:46.766600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:46.766600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:46.766600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:46.766600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:46.766600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:46.766600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:46.766600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:46.766600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:46.766600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 15:02:46.766600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 15:02:46.766600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 15:02:46.766600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 15:02:46.767676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:46.767676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:46.767676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:46.767676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:46.767676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:46.767676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:46.767676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:46.767676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:46.767676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:46.767676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:46.767676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:46.767676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:46.767676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:46.767676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:46.767676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:46.767676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:46.767676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 15:02:46.768584 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 15:02:46.768584 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:02:46.768584 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:02:46.768584 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 622/1251] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:02:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:02:46.774338 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5d6e2-a97d-46c5-a555-b94b279eb363 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:46.927135 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5d6e2-a97d-46c5-a555-b94b279eb363 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:46.968874 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f84fad2-3ad2-420f-9c1b-bcf033e2356b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:46.969516 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84fad2-3ad2-420f-9c1b-bcf033e2356b None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:46.969791 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84fad2-3ad2-420f-9c1b-bcf033e2356b None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:46.970154 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84fad2-3ad2-420f-9c1b-bcf033e2356b None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:46.970714 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f84fad2-3ad2-420f-9c1b-bcf033e2356b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:46.971013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f84fad2-3ad2-420f-9c1b-bcf033e2356b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:46.972410 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f84fad2-3ad2-420f-9c1b-bcf033e2356b None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:46.986882 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 630/1252] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 15:02:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:02:46.993935 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cca6fc-d495-401c-ba7f-6ece85eb4fc5 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:47.134103 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cca6fc-d495-401c-ba7f-6ece85eb4fc5 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:47.178770 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:47.189183 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa40fb3-0e80-44ee-a7f7-5a02dfd728b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:47.190123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa40fb3-0e80-44ee-a7f7-5a02dfd728b5 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:47.190406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa40fb3-0e80-44ee-a7f7-5a02dfd728b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:47.191357 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa40fb3-0e80-44ee-a7f7-5a02dfd728b5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:47.202471 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa40fb3-0e80-44ee-a7f7-5a02dfd728b5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:47.204899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa40fb3-0e80-44ee-a7f7-5a02dfd728b5 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:47.221771 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-baa40fb3-0e80-44ee-a7f7-5a02dfd728b5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 15:02:47.221771 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:47.221771 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:47.221771 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:47.221771 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:47.221771 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:47.221771 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:47.221771 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:47.221771 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:47.221771 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:47.221771 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:47.221771 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:47.221771 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 15:02:47.221771 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 15:02:47.221771 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 15:02:47.221771 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 15:02:47.223184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:47.223184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:47.223184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 15:02:47.223184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 15:02:47.223184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 15:02:47.223184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 15:02:47.223184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:47.223184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:47.223184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:47.223184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:47.223184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:47.223184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:47.223184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:47.223184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:47.223184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:47.223184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:47.223184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:47.223184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:47.225511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:47.225511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:47.225511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 15:02:47.225511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 15:02:47.225511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 15:02:47.225511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 15:02:47.225511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 15:02:47.225511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 15:02:47.225511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 15:02:47.225511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:02:47.225511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 623/1253] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:02:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:02:47.231093 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f84fad2-3ad2-420f-9c1b-bcf033e2356b None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:47.314837 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:47.371763 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f84fad2-3ad2-420f-9c1b-bcf033e2356b None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:47.424004 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41e645-243f-4ee3-a971-10aa5fe7805c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:47.424625 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41e645-243f-4ee3-a971-10aa5fe7805c None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:47.424897 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41e645-243f-4ee3-a971-10aa5fe7805c None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:47.425191 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41e645-243f-4ee3-a971-10aa5fe7805c None admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:47.425973 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41e645-243f-4ee3-a971-10aa5fe7805c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:47.426317 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41e645-243f-4ee3-a971-10aa5fe7805c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:47.427877 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41e645-243f-4ee3-a971-10aa5fe7805c None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:47.458298 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 631/1254] 10.222.0.44 () {56 vars in 1143 bytes} [Sun Sep 11 15:02:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:02:47.464971 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa40fb3-0e80-44ee-a7f7-5a02dfd728b5 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:47.594200 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa40fb3-0e80-44ee-a7f7-5a02dfd728b5 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:47.640590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9621f5-5820-4937-b9ea-bb639f86d344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:47.641500 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9621f5-5820-4937-b9ea-bb639f86d344 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:47.641992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9621f5-5820-4937-b9ea-bb639f86d344 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:47.642164 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9621f5-5820-4937-b9ea-bb639f86d344 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:47.720237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:47.737400 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9621f5-5820-4937-b9ea-bb639f86d344 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:47.738866 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9621f5-5820-4937-b9ea-bb639f86d344 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:47.883909 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 624/1255] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 15:02:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 15:02:47.891824 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41e645-243f-4ee3-a971-10aa5fe7805c None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:48.040686 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41e645-243f-4ee3-a971-10aa5fe7805c None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:48.058632 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:48.094415 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54574a86-7d15-44d6-ba15-4a1c3c05924f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:48.095081 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54574a86-7d15-44d6-ba15-4a1c3c05924f None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:48.095356 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54574a86-7d15-44d6-ba15-4a1c3c05924f None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:48.095622 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54574a86-7d15-44d6-ba15-4a1c3c05924f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:48.109274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54574a86-7d15-44d6-ba15-4a1c3c05924f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:48.110780 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54574a86-7d15-44d6-ba15-4a1c3c05924f None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:48.129716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-54574a86-7d15-44d6-ba15-4a1c3c05924f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:02:48.129716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:48.129716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:48.129716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:48.129716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:48.129716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:48.129716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:48.129716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:48.129716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:48.129716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:48.129716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:48.129716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:48.129716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 15:02:48.129716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 15:02:48.129716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 15:02:48.129716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 15:02:48.130952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:48.130952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:48.130952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:48.130952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:48.130952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:48.130952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:48.130952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:48.130952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:48.130952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:48.130952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:48.130952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:48.130952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:48.130952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:48.130952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:48.130952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:48.130952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:48.130952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 15:02:48.132146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 15:02:48.132146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:02:48.132146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:02:48.132146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 632/1256] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:02:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 15:02:48.141209 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9621f5-5820-4937-b9ea-bb639f86d344 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:48.242062 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:48.259590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9621f5-5820-4937-b9ea-bb639f86d344 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:48.304879 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98e5aa-8965-4aed-a012-8d3d2beff7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:48.305431 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98e5aa-8965-4aed-a012-8d3d2beff7ca None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:48.305701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98e5aa-8965-4aed-a012-8d3d2beff7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:48.305962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98e5aa-8965-4aed-a012-8d3d2beff7ca None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:48.306606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98e5aa-8965-4aed-a012-8d3d2beff7ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:48.306898 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98e5aa-8965-4aed-a012-8d3d2beff7ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:48.308288 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98e5aa-8965-4aed-a012-8d3d2beff7ca None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:48.321034 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 625/1257] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 15:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:02:48.327446 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54574a86-7d15-44d6-ba15-4a1c3c05924f None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:48.448043 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54574a86-7d15-44d6-ba15-4a1c3c05924f None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:48.486857 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8386f3-4c88-496d-882f-18dcdc2c1558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:48.487615 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8386f3-4c88-496d-882f-18dcdc2c1558 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:48.487890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8386f3-4c88-496d-882f-18dcdc2c1558 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:48.488273 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8386f3-4c88-496d-882f-18dcdc2c1558 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:48.505486 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8386f3-4c88-496d-882f-18dcdc2c1558 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:48.507334 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8386f3-4c88-496d-882f-18dcdc2c1558 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:48.522846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-4d8386f3-4c88-496d-882f-18dcdc2c1558 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 15:02:48.522846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:48.522846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:48.522846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:48.522846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:48.522846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:48.522846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:48.522846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:48.522846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:48.522846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:48.522846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:48.522846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:48.522846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 15:02:48.522846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 15:02:48.522846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 15:02:48.524148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 15:02:48.524148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:48.524148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:48.524148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:48.524148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:48.524148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:48.524148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:48.524148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:48.524148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:48.524148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:48.524148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:48.524148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:48.524148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:48.524148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:48.524148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:48.524148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:48.524148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:48.525572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 15:02:48.525572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 15:02:48.525572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 15:02:48.525572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 15:02:48.525572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 15:02:48.525572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 15:02:48.525572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 15:02:48.525572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:02:48.525572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 633/1258] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 15:02:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 15:02:48.530497 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98e5aa-8965-4aed-a012-8d3d2beff7ca None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:48.662210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98e5aa-8965-4aed-a012-8d3d2beff7ca None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:48.702957 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e0991-0a9c-47fa-8cba-0fe62d160540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:48.703582 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e0991-0a9c-47fa-8cba-0fe62d160540 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:48.703855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e0991-0a9c-47fa-8cba-0fe62d160540 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:48.704115 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e0991-0a9c-47fa-8cba-0fe62d160540 None admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:48.704808 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e0991-0a9c-47fa-8cba-0fe62d160540 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:48.705107 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e0991-0a9c-47fa-8cba-0fe62d160540 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:48.706624 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e0991-0a9c-47fa-8cba-0fe62d160540 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:48.720314 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 626/1259] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 15:02:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:02:48.727104 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8386f3-4c88-496d-882f-18dcdc2c1558 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:48.755250 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:48.845331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8386f3-4c88-496d-882f-18dcdc2c1558 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:48.894235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea4175f-ce8e-4c6d-8236-9e8279a5cfa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:48.894943 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4175f-ce8e-4c6d-8236-9e8279a5cfa4 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:48.895136 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4175f-ce8e-4c6d-8236-9e8279a5cfa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:48.895404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4175f-ce8e-4c6d-8236-9e8279a5cfa4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:48.896139 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea4175f-ce8e-4c6d-8236-9e8279a5cfa4 None admin] RBAC: Adding query filter params (role.id=c426875a15834aa5b122e2a62267d15a, scope.project.id=70ac515938a24c45ae6302dbac46a1ab, user.id=ead06362f8a2444bbfe1405cd602f1c8) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:48.896442 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea4175f-ce8e-4c6d-8236-9e8279a5cfa4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:48.898033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea4175f-ce8e-4c6d-8236-9e8279a5cfa4 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:48.904457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 634/1260] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 15:02:48 2022] GET /identity/v3/role_assignments?user.id=ead06362f8a2444bbfe1405cd602f1c8&scope.project.id=70ac515938a24c45ae6302dbac46a1ab&role.id=c426875a15834aa5b122e2a62267d15a => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:02:51.035384 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad1afb-69e4-4aab-b7e1-77ab49b33ca3 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:51.036013 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad1afb-69e4-4aab-b7e1-77ab49b33ca3 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:51.036013 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad1afb-69e4-4aab-b7e1-77ab49b33ca3 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:51.037347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 627/1261] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:02:51.042917 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bd8eb-fcb1-4257-90f8-ae4655144047 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:51.043426 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bd8eb-fcb1-4257-90f8-ae4655144047 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:51.043426 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bd8eb-fcb1-4257-90f8-ae4655144047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:51.063715 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-9b4bd8eb-fcb1-4257-90f8-ae4655144047 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:02:51.064941 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-9b4bd8eb-fcb1-4257-90f8-ae4655144047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:02:51.140762 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 635/1262] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:02:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:02:51.410917 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad1afb-69e4-4aab-b7e1-77ab49b33ca3 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:51.422456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:51.519839 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad1afb-69e4-4aab-b7e1-77ab49b33ca3 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:51.554884 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28022c61-926e-4516-b8a2-2f8dd2ae7563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:51.555512 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28022c61-926e-4516-b8a2-2f8dd2ae7563 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:51.555784 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28022c61-926e-4516-b8a2-2f8dd2ae7563 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:51.556043 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28022c61-926e-4516-b8a2-2f8dd2ae7563 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:51.566643 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28022c61-926e-4516-b8a2-2f8dd2ae7563 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 15:02:51.571408 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-28022c61-926e-4516-b8a2-2f8dd2ae7563 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 15:02:51.571408 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:51.571408 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:51.571408 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:51.571408 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:51.571408 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:51.571408 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:51.571408 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:51.571408 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:51.571408 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:51.571408 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:51.571408 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:51.571408 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 15:02:51.571408 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 15:02:51.571408 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 15:02:51.571408 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 15:02:51.572474 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 15:02:51.572474 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 15:02:51.572474 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 15:02:51.572474 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 15:02:51.572474 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:51.572474 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:51.572474 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:51.572474 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:51.572474 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:51.572474 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:51.572474 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:51.572474 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:51.572474 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:51.572474 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:51.572474 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:51.572474 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:51.573476 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:51.573476 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:51.573476 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:51.573476 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:51.573476 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 15:02:51.573476 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 15:02:51.573476 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 15:02:51.573476 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 15:02:51.573476 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 15:02:51.573476 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 15:02:51.573476 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 15:02:51.573476 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:02:51.573476 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 628/1263] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 15:02:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:02:51.578860 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bd8eb-fcb1-4257-90f8-ae4655144047 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:51.598527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:51.683728 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bd8eb-fcb1-4257-90f8-ae4655144047 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:51.718976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97eae50-4205-4f6e-b68f-0835fd4d844d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:51.719569 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97eae50-4205-4f6e-b68f-0835fd4d844d None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:51.719844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97eae50-4205-4f6e-b68f-0835fd4d844d None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:51.720114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97eae50-4205-4f6e-b68f-0835fd4d844d None admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:51.720772 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97eae50-4205-4f6e-b68f-0835fd4d844d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:51.721072 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97eae50-4205-4f6e-b68f-0835fd4d844d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:51.722479 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97eae50-4205-4f6e-b68f-0835fd4d844d None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:51.731493 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 636/1264] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 15:02:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:02:51.737822 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28022c61-926e-4516-b8a2-2f8dd2ae7563 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:51.813305 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:51.854687 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28022c61-926e-4516-b8a2-2f8dd2ae7563 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:51.894855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d47ca-5556-4be3-ba76-bc5ca4f4b732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:51.895491 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d47ca-5556-4be3-ba76-bc5ca4f4b732 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:51.895757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d47ca-5556-4be3-ba76-bc5ca4f4b732 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:51.896020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d47ca-5556-4be3-ba76-bc5ca4f4b732 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:51.908012 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d47ca-5556-4be3-ba76-bc5ca4f4b732 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:51.909557 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d47ca-5556-4be3-ba76-bc5ca4f4b732 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:51.926064 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-2c5d47ca-5556-4be3-ba76-bc5ca4f4b732 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:02:51.926064 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:51.926064 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:51.926064 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:51.926064 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:51.926064 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:51.926064 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:51.926064 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:51.926064 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:51.926064 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:51.926064 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:51.926064 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:51.926064 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 15:02:51.926064 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 15:02:51.926064 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 15:02:51.926064 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 15:02:51.927267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:51.927267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:51.927267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:51.927267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:51.927267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:51.927267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:51.927267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:51.927267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:51.927267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:51.927267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:51.927267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:51.927267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:51.927267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:51.927267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:51.927267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:51.927267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:51.927267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 15:02:51.928227 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 15:02:51.928227 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:02:51.928227 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:02:51.928227 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 629/1265] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:02:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:02:51.933534 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97eae50-4205-4f6e-b68f-0835fd4d844d None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:52.026417 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:52.047009 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97eae50-4205-4f6e-b68f-0835fd4d844d None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:52.081930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1bb5e-a29f-4b63-8f66-9ea63052e6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:52.082540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1bb5e-a29f-4b63-8f66-9ea63052e6a9 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:52.082817 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1bb5e-a29f-4b63-8f66-9ea63052e6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:52.083088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1bb5e-a29f-4b63-8f66-9ea63052e6a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:52.083728 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b1bb5e-a29f-4b63-8f66-9ea63052e6a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:52.084029 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b1bb5e-a29f-4b63-8f66-9ea63052e6a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:52.085464 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b1bb5e-a29f-4b63-8f66-9ea63052e6a9 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:52.098410 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 637/1266] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 15:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:02:52.104462 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d47ca-5556-4be3-ba76-bc5ca4f4b732 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:52.211463 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d47ca-5556-4be3-ba76-bc5ca4f4b732 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:52.237360 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:52.252843 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446e8c23-d1e9-4b2e-b4a7-6779ce796d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:52.253512 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e8c23-d1e9-4b2e-b4a7-6779ce796d95 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:52.253788 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e8c23-d1e9-4b2e-b4a7-6779ce796d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:52.254062 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e8c23-d1e9-4b2e-b4a7-6779ce796d95 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:52.262131 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446e8c23-d1e9-4b2e-b4a7-6779ce796d95 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:52.263580 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446e8c23-d1e9-4b2e-b4a7-6779ce796d95 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:52.274965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-446e8c23-d1e9-4b2e-b4a7-6779ce796d95 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 15:02:52.274965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:52.274965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:52.274965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:52.274965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:52.274965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:52.274965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:52.274965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:52.274965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:52.274965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:52.274965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:52.274965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:52.274965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 15:02:52.274965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 15:02:52.274965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 15:02:52.274965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 15:02:52.276203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:52.276203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:52.276203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 15:02:52.276203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 15:02:52.276203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 15:02:52.276203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 15:02:52.276203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:52.276203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:52.276203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:52.276203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:52.276203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:52.276203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:52.276203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:52.276203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:52.276203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:52.276203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:52.276203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:52.276203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:52.277798 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:52.277798 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:52.277798 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 15:02:52.277798 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 15:02:52.277798 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 15:02:52.277798 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 15:02:52.277798 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 15:02:52.277798 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 15:02:52.277798 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 15:02:52.277798 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:02:52.277798 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 630/1267] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:02:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:02:52.283206 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1bb5e-a29f-4b63-8f66-9ea63052e6a9 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:52.387685 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1bb5e-a29f-4b63-8f66-9ea63052e6a9 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:52.422319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2bb225-ae8d-4eb0-ab03-322f8a7c87b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:52.422920 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2bb225-ae8d-4eb0-ab03-322f8a7c87b0 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:52.423163 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2bb225-ae8d-4eb0-ab03-322f8a7c87b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:52.423453 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2bb225-ae8d-4eb0-ab03-322f8a7c87b0 None admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:52.424163 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2bb225-ae8d-4eb0-ab03-322f8a7c87b0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:52.424447 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2bb225-ae8d-4eb0-ab03-322f8a7c87b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:52.425896 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2bb225-ae8d-4eb0-ab03-322f8a7c87b0 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:52.434877 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:52.443385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 638/1268] 10.222.0.44 () {56 vars in 1143 bytes} [Sun Sep 11 15:02:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:02:52.449682 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446e8c23-d1e9-4b2e-b4a7-6779ce796d95 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:52.557249 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446e8c23-d1e9-4b2e-b4a7-6779ce796d95 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:52.591147 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b0ef59-3266-43b4-afed-d1cc79e85b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:52.592134 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0ef59-3266-43b4-afed-d1cc79e85b50 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:52.592410 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0ef59-3266-43b4-afed-d1cc79e85b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:52.592673 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0ef59-3266-43b4-afed-d1cc79e85b50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:52.603571 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b0ef59-3266-43b4-afed-d1cc79e85b50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:52.605014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b0ef59-3266-43b4-afed-d1cc79e85b50 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:52.619376 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-f6b0ef59-3266-43b4-afed-d1cc79e85b50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:02:52.619376 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:52.619376 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:52.619376 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:52.619376 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:52.619376 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:52.619376 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:52.619376 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:52.619376 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:52.619376 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:52.619376 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:52.619376 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:52.619376 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 15:02:52.619376 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 15:02:52.619376 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 15:02:52.619376 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 15:02:52.620406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:52.620406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:52.620406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:52.620406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:52.620406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:52.620406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:52.620406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:52.620406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:52.620406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:52.620406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:52.620406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:52.620406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:52.620406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:52.620406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:52.620406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:52.620406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:52.620406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 15:02:52.621513 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 15:02:52.621513 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:02:52.621513 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:02:52.621513 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 631/1269] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:02:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:02:52.626746 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2bb225-ae8d-4eb0-ab03-322f8a7c87b0 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:52.731292 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2bb225-ae8d-4eb0-ab03-322f8a7c87b0 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:52.768161 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf21d6-9734-46d7-99bd-091e79a554cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:52.768764 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf21d6-9734-46d7-99bd-091e79a554cd None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:52.769036 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf21d6-9734-46d7-99bd-091e79a554cd None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:52.769334 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf21d6-9734-46d7-99bd-091e79a554cd None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:52.769972 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf21d6-9734-46d7-99bd-091e79a554cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:52.770271 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf21d6-9734-46d7-99bd-091e79a554cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:52.771665 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf21d6-9734-46d7-99bd-091e79a554cd None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:52.783008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 639/1270] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 15:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:02:52.789326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b0ef59-3266-43b4-afed-d1cc79e85b50 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:52.823020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:52.902880 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b0ef59-3266-43b4-afed-d1cc79e85b50 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:52.938290 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36492b44-0a8a-4f1b-be0f-81c77705b92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:52.938922 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36492b44-0a8a-4f1b-be0f-81c77705b92f None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:52.939195 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36492b44-0a8a-4f1b-be0f-81c77705b92f None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:52.939455 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36492b44-0a8a-4f1b-be0f-81c77705b92f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:52.946795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36492b44-0a8a-4f1b-be0f-81c77705b92f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:52.948261 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36492b44-0a8a-4f1b-be0f-81c77705b92f None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:52.960081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-36492b44-0a8a-4f1b-be0f-81c77705b92f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 15:02:52.960081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:52.960081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:52.960081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:52.960081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:52.960081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:52.960081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:52.960081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:52.960081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:52.960081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:52.960081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:52.960081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:52.960081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 15:02:52.960081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 15:02:52.960081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 15:02:52.961123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 15:02:52.961123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:52.961123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:52.961123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:52.961123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:52.961123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:52.961123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:52.961123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:52.961123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:52.961123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:52.961123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:52.961123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:52.961123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:52.961123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:52.961123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:52.961123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:52.961123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:52.966219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 15:02:52.966219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 15:02:52.966219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 15:02:52.966219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 15:02:52.966219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 15:02:52.966219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 15:02:52.966219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 15:02:52.966219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:02:52.966219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 632/1271] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 15:02:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:02:52.967753 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf21d6-9734-46d7-99bd-091e79a554cd None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:53.031114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:53.073733 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf21d6-9734-46d7-99bd-091e79a554cd None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:53.109165 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9caf5-e3fb-4656-8899-67b62083da75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:53.109829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9caf5-e3fb-4656-8899-67b62083da75 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:53.110099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9caf5-e3fb-4656-8899-67b62083da75 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:53.110352 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9caf5-e3fb-4656-8899-67b62083da75 None admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:53.111028 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9caf5-e3fb-4656-8899-67b62083da75 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:53.111332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9caf5-e3fb-4656-8899-67b62083da75 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:53.112748 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9caf5-e3fb-4656-8899-67b62083da75 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:53.125210 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 640/1272] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 15:02:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:02:53.131728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36492b44-0a8a-4f1b-be0f-81c77705b92f None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:53.226870 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:53.236499 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36492b44-0a8a-4f1b-be0f-81c77705b92f None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:53.271865 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9644c-fb05-40b7-a754-e3bd0a81b99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:53.272530 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9644c-fb05-40b7-a754-e3bd0a81b99a None admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:53.272809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9644c-fb05-40b7-a754-e3bd0a81b99a None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:53.273073 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9644c-fb05-40b7-a754-e3bd0a81b99a None admin] PATH_INFO: `/v3/projects/70ac515938a24c45ae6302dbac46a1ab/users/ead06362f8a2444bbfe1405cd602f1c8/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:53.276950 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9644c-fb05-40b7-a754-e3bd0a81b99a None admin] RBAC: Authorizing `identity:create_grant(project_id=70ac515938a24c45ae6302dbac46a1ab, user_id=ead06362f8a2444bbfe1405cd602f1c8, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:53.277379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9644c-fb05-40b7-a754-e3bd0a81b99a None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:53.292255 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 633/1273] 10.222.0.44 () {58 vars in 1439 bytes} [Sun Sep 11 15:02:53 2022] PUT /identity/v3/projects/70ac515938a24c45ae6302dbac46a1ab/users/ead06362f8a2444bbfe1405cd602f1c8/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:02:55.412073 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ac8cf-5de2-4e5c-ba45-e6dada2ab2ac None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:55.412627 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ac8cf-5de2-4e5c-ba45-e6dada2ab2ac None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:55.412627 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ac8cf-5de2-4e5c-ba45-e6dada2ab2ac None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:55.414014 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 641/1274] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:02:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:02:55.419274 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041b974-fd41-45f9-8239-ec31853abf6a None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:55.419755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041b974-fd41-45f9-8239-ec31853abf6a None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:55.419857 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041b974-fd41-45f9-8239-ec31853abf6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:55.439483 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-6041b974-fd41-45f9-8239-ec31853abf6a None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:02:55.440704 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-6041b974-fd41-45f9-8239-ec31853abf6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:02:55.515277 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 634/1275] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:02:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:02:55.778239 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ac8cf-5de2-4e5c-ba45-e6dada2ab2ac None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:55.891746 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ac8cf-5de2-4e5c-ba45-e6dada2ab2ac None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:55.912824 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:55.932837 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c5e8c-a01a-4dad-ac99-09c209998f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:55.933526 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c5e8c-a01a-4dad-ac99-09c209998f8b None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:55.933920 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c5e8c-a01a-4dad-ac99-09c209998f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:55.934273 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c5e8c-a01a-4dad-ac99-09c209998f8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:55.945388 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c5e8c-a01a-4dad-ac99-09c209998f8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 15:02:55.950150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-321c5e8c-a01a-4dad-ac99-09c209998f8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 15:02:55.950150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:55.950150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:55.950150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:55.950150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:55.950150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:55.950150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:55.950150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:55.950150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:55.950150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:55.950150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:55.950150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:55.950150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 15:02:55.950150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 15:02:55.950150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 15:02:55.950150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 15:02:55.951300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 15:02:55.951300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 15:02:55.951300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 15:02:55.951300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 15:02:55.951300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:55.951300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:55.951300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:55.951300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:55.951300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:55.951300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:55.951300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:55.951300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:55.951300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:55.951300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:55.951300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:55.951300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:55.952127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:55.952127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:55.952127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:55.952127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:55.952127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 15:02:55.952127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 15:02:55.952127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 15:02:55.952127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 15:02:55.952127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 15:02:55.952127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 15:02:55.952127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 15:02:55.952127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:02:55.952127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 642/1276] 10.222.0.44 () {56 vars in 1096 bytes} [Sun Sep 11 15:02:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:02:55.957411 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041b974-fd41-45f9-8239-ec31853abf6a None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:56.039469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:56.060188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041b974-fd41-45f9-8239-ec31853abf6a None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:56.095928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71874f7e-4ffd-4a72-bced-f1ec45cbb44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:56.096530 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71874f7e-4ffd-4a72-bced-f1ec45cbb44e None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:56.096803 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71874f7e-4ffd-4a72-bced-f1ec45cbb44e None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:56.097069 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71874f7e-4ffd-4a72-bced-f1ec45cbb44e None admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:56.097760 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71874f7e-4ffd-4a72-bced-f1ec45cbb44e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:56.098060 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71874f7e-4ffd-4a72-bced-f1ec45cbb44e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:56.099438 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71874f7e-4ffd-4a72-bced-f1ec45cbb44e None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:56.107973 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 635/1277] 10.222.0.44 () {56 vars in 1099 bytes} [Sun Sep 11 15:02:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:02:56.114118 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c5e8c-a01a-4dad-ac99-09c209998f8b None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:56.221069 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c5e8c-a01a-4dad-ac99-09c209998f8b None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:56.259591 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61258c-ce68-4c81-9224-bee82f53c8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:56.260481 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61258c-ce68-4c81-9224-bee82f53c8ee None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:56.260752 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61258c-ce68-4c81-9224-bee82f53c8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:56.261012 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61258c-ce68-4c81-9224-bee82f53c8ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:56.271924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a61258c-ce68-4c81-9224-bee82f53c8ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:56.273421 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a61258c-ce68-4c81-9224-bee82f53c8ee None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:56.289095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-3a61258c-ce68-4c81-9224-bee82f53c8ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:02:56.289095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:56.289095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:56.289095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:56.289095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:56.289095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:56.289095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:56.289095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:56.289095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:56.289095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:56.289095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:56.289095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:56.289095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 15:02:56.289095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 15:02:56.289095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 15:02:56.289095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 15:02:56.290315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:56.290315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:56.290315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:56.290315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:56.290315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:56.290315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:56.290315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:56.290315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:56.290315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:56.290315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:56.290315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:56.290315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:56.290315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:56.290315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:56.290315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:56.290315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:56.290315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 15:02:56.291595 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 15:02:56.291595 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:02:56.291595 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:02:56.291595 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 643/1278] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:02:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:02:56.296139 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71874f7e-4ffd-4a72-bced-f1ec45cbb44e None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:56.404674 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71874f7e-4ffd-4a72-bced-f1ec45cbb44e None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:56.440317 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce0239f-e598-444e-8ccd-04f848a5c91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:56.441016 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0239f-e598-444e-8ccd-04f848a5c91f None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:56.441170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0239f-e598-444e-8ccd-04f848a5c91f None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:56.441459 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce0239f-e598-444e-8ccd-04f848a5c91f None admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:56.442085 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce0239f-e598-444e-8ccd-04f848a5c91f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:56.442374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce0239f-e598-444e-8ccd-04f848a5c91f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:56.443745 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce0239f-e598-444e-8ccd-04f848a5c91f None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:56.455775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 636/1279] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 15:02:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:02:56.461790 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61258c-ce68-4c81-9224-bee82f53c8ee None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:56.494699 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:56.563739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61258c-ce68-4c81-9224-bee82f53c8ee None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:56.598243 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154cbb2-5c80-44f3-a335-8dadf6971ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:56.598870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154cbb2-5c80-44f3-a335-8dadf6971ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:56.599139 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154cbb2-5c80-44f3-a335-8dadf6971ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:56.599409 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154cbb2-5c80-44f3-a335-8dadf6971ae3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:56.608770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2154cbb2-5c80-44f3-a335-8dadf6971ae3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:56.610244 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2154cbb2-5c80-44f3-a335-8dadf6971ae3 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:56.622088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-2154cbb2-5c80-44f3-a335-8dadf6971ae3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 15:02:56.622088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:56.622088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:56.622088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:56.622088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:56.622088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:56.622088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:56.622088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:56.622088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:56.622088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:56.622088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:56.622088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:56.622088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 15:02:56.622088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 15:02:56.622088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 15:02:56.622088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 15:02:56.623262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:56.623262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:56.623262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 15:02:56.623262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 15:02:56.623262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 15:02:56.623262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 15:02:56.623262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:56.623262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:56.623262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:56.623262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:56.623262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:56.623262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:56.623262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:56.623262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:56.623262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:56.623262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:56.623262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:56.623262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:56.624402 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:56.624402 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:56.624402 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 15:02:56.624402 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 15:02:56.624402 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 15:02:56.624402 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 15:02:56.624402 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 15:02:56.624402 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 15:02:56.624402 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 15:02:56.624402 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:02:56.624402 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 644/1280] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:02:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:02:56.629606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce0239f-e598-444e-8ccd-04f848a5c91f None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:56.633864 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:56.738575 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce0239f-e598-444e-8ccd-04f848a5c91f None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:56.774484 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7d524-0f0b-4e1c-9acb-bc5e782527fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:56.775094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7d524-0f0b-4e1c-9acb-bc5e782527fa None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:56.775372 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7d524-0f0b-4e1c-9acb-bc5e782527fa None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:56.775639 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7d524-0f0b-4e1c-9acb-bc5e782527fa None admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:56.776369 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb7d524-0f0b-4e1c-9acb-bc5e782527fa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:56.776672 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb7d524-0f0b-4e1c-9acb-bc5e782527fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:56.778141 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb7d524-0f0b-4e1c-9acb-bc5e782527fa None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:56.795722 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 637/1281] 10.222.0.44 () {56 vars in 1143 bytes} [Sun Sep 11 15:02:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:02:56.801871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154cbb2-5c80-44f3-a335-8dadf6971ae3 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:56.903543 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:56.914931 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154cbb2-5c80-44f3-a335-8dadf6971ae3 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:56.961682 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0677291b-34cd-4f41-91fe-e517a01444da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:56.962293 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0677291b-34cd-4f41-91fe-e517a01444da None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:56.962562 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0677291b-34cd-4f41-91fe-e517a01444da None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:56.962826 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0677291b-34cd-4f41-91fe-e517a01444da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:57.033361 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0677291b-34cd-4f41-91fe-e517a01444da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:57.034771 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0677291b-34cd-4f41-91fe-e517a01444da None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:57.123419 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:57.134137 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 645/1282] 10.222.0.44 () {58 vars in 1282 bytes} [Sun Sep 11 15:02:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 15:02:57.140158 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7d524-0f0b-4e1c-9acb-bc5e782527fa None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:57.201624 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:57.248082 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7d524-0f0b-4e1c-9acb-bc5e782527fa None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:57.285862 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0b0d6-79cf-4378-9c78-5637cc8406ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:57.286494 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0b0d6-79cf-4378-9c78-5637cc8406ab None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:57.286801 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0b0d6-79cf-4378-9c78-5637cc8406ab None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:57.287116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0b0d6-79cf-4378-9c78-5637cc8406ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:57.297554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0b0d6-79cf-4378-9c78-5637cc8406ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:57.299021 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0b0d6-79cf-4378-9c78-5637cc8406ab None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:57.313497 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-3ae0b0d6-79cf-4378-9c78-5637cc8406ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:02:57.313497 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:57.313497 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:57.313497 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:57.313497 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:57.313497 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:57.313497 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:57.313497 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:57.313497 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:57.313497 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:57.313497 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:57.313497 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:57.313497 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 15:02:57.313497 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 15:02:57.313497 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 15:02:57.313497 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 15:02:57.314554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:57.314554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:57.314554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:57.314554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:57.314554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:57.314554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:57.314554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:57.314554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:57.314554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:57.314554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:57.314554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:57.314554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:57.314554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:57.314554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:57.314554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:57.314554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:57.314554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 15:02:57.315444 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 15:02:57.315444 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 15:02:57.315444 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:02:57.315444 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 638/1283] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:02:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:02:57.320742 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0677291b-34cd-4f41-91fe-e517a01444da None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:57.423859 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0677291b-34cd-4f41-91fe-e517a01444da None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:57.463934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f65606-f0b6-4aac-9f69-40e8ae564985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:57.464619 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f65606-f0b6-4aac-9f69-40e8ae564985 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:57.464729 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f65606-f0b6-4aac-9f69-40e8ae564985 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:57.464978 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f65606-f0b6-4aac-9f69-40e8ae564985 None admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:57.465669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f65606-f0b6-4aac-9f69-40e8ae564985 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:57.465964 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f65606-f0b6-4aac-9f69-40e8ae564985 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:57.467738 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f65606-f0b6-4aac-9f69-40e8ae564985 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:57.480636 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 646/1284] 10.222.0.44 () {56 vars in 1109 bytes} [Sun Sep 11 15:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:02:57.486625 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0b0d6-79cf-4378-9c78-5637cc8406ab None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:57.594409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0b0d6-79cf-4378-9c78-5637cc8406ab None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:57.608594 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:57.630413 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cdd82b-f322-4eca-a4fa-6dc50b8b6bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:57.631184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cdd82b-f322-4eca-a4fa-6dc50b8b6bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:57.631461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cdd82b-f322-4eca-a4fa-6dc50b8b6bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:57.631726 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cdd82b-f322-4eca-a4fa-6dc50b8b6bd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:57.638827 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cdd82b-f322-4eca-a4fa-6dc50b8b6bd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:57.640285 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cdd82b-f322-4eca-a4fa-6dc50b8b6bd2 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:57.651899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application [None req-20cdd82b-f322-4eca-a4fa-6dc50b8b6bd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 15:02:57.651899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:02:57.651899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:02:57.651899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:02:57.651899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:02:57.651899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:02:57.651899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:02:57.651899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:02:57.651899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:02:57.651899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:02:57.651899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:02:57.651899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:02:57.651899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 15:02:57.651899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 15:02:57.651899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 15:02:57.652936 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 15:02:57.652936 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:02:57.652936 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:02:57.652936 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:02:57.652936 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:02:57.652936 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:02:57.652936 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:02:57.652936 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:02:57.652936 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application with Lock( Sep 11 15:02:57.652936 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:02:57.652936 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:02:57.652936 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:02:57.652936 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:02:57.652936 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:02:57.652936 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:02:57.652936 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:02:57.652936 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:02:57.654074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 15:02:57.654074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 15:02:57.654074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 15:02:57.654074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 15:02:57.654074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 15:02:57.654074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 15:02:57.654074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 15:02:57.654074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: ERROR keystone.server.flask.application Sep 11 15:02:57.654074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 639/1285] 10.222.0.44 () {56 vars in 1111 bytes} [Sun Sep 11 15:02:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:02:57.659980 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f65606-f0b6-4aac-9f69-40e8ae564985 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:57.711216 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:02:57.763213 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f65606-f0b6-4aac-9f69-40e8ae564985 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:57.797781 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66adf9a8-0e4f-4eef-b084-a631f143b09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:57.798379 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adf9a8-0e4f-4eef-b084-a631f143b09f None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:57.798646 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adf9a8-0e4f-4eef-b084-a631f143b09f None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:57.798909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adf9a8-0e4f-4eef-b084-a631f143b09f None admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:57.799589 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66adf9a8-0e4f-4eef-b084-a631f143b09f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:57.799886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66adf9a8-0e4f-4eef-b084-a631f143b09f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:57.801377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66adf9a8-0e4f-4eef-b084-a631f143b09f None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:57.813489 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 647/1286] 10.222.0.44 () {56 vars in 1148 bytes} [Sun Sep 11 15:02:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:02:57.819973 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cdd82b-f322-4eca-a4fa-6dc50b8b6bd2 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:02:57.922303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cdd82b-f322-4eca-a4fa-6dc50b8b6bd2 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:02:57.957900 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f48369-9dbe-4bae-a990-56b0dc0de3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:02:57.958502 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f48369-9dbe-4bae-a990-56b0dc0de3b6 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:02:57.958775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f48369-9dbe-4bae-a990-56b0dc0de3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:02:57.959130 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f48369-9dbe-4bae-a990-56b0dc0de3b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:02:57.959865 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f48369-9dbe-4bae-a990-56b0dc0de3b6 None admin] RBAC: Adding query filter params (role.id=c426875a15834aa5b122e2a62267d15a, scope.project.id=70ac515938a24c45ae6302dbac46a1ab, user.id=ead06362f8a2444bbfe1405cd602f1c8) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:02:57.960171 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f48369-9dbe-4bae-a990-56b0dc0de3b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:02:57.961657 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f48369-9dbe-4bae-a990-56b0dc0de3b6 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:02:57.967938 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 640/1287] 10.222.0.44 () {56 vars in 1374 bytes} [Sun Sep 11 15:02:57 2022] GET /identity/v3/role_assignments?user.id=ead06362f8a2444bbfe1405cd602f1c8&scope.project.id=70ac515938a24c45ae6302dbac46a1ab&role.id=c426875a15834aa5b122e2a62267d15a => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:03:00.142593 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e9744-3c0f-4940-b90e-313be3901f20 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:00.143156 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e9744-3c0f-4940-b90e-313be3901f20 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:00.143156 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e9744-3c0f-4940-b90e-313be3901f20 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:00.144450 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 648/1288] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:03:00.149989 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348efe6d-3822-4635-99d9-9fd3a3b0dbac None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:00.150444 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348efe6d-3822-4635-99d9-9fd3a3b0dbac None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:00.150538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348efe6d-3822-4635-99d9-9fd3a3b0dbac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:00.169540 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-348efe6d-3822-4635-99d9-9fd3a3b0dbac None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:03:00.170745 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-348efe6d-3822-4635-99d9-9fd3a3b0dbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:03:00.202565 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:03:00.246268 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 641/1289] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:03:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:03:00.516668 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e9744-3c0f-4940-b90e-313be3901f20 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:03:00.633634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e9744-3c0f-4940-b90e-313be3901f20 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:03:00.645947 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:03:00.671943 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e32680-6e1f-4986-9f82-80da69574ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:03:00.672595 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e32680-6e1f-4986-9f82-80da69574ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:00.672876 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e32680-6e1f-4986-9f82-80da69574ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:00.673161 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e32680-6e1f-4986-9f82-80da69574ae1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:00.673765 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e32680-6e1f-4986-9f82-80da69574ae1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:03:00.675150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e32680-6e1f-4986-9f82-80da69574ae1 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:03:00.684618 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-a2e32680-6e1f-4986-9f82-80da69574ae1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 15:03:00.684618 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:03:00.684618 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 15:03:00.684618 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 15:03:00.684618 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 15:03:00.684618 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 15:03:00.684618 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 15:03:00.684618 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:03:00.684618 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 15:03:00.684618 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:03:00.684618 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 15:03:00.684618 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:03:00.684618 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:03:00.684618 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:03:00.684618 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:03:00.684618 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:03:00.684618 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:03:00.685936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:03:00.685936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:03:00.685936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:03:00.685936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:03:00.685936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:03:00.685936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:03:00.685936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 15:03:00.685936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 15:03:00.685936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 15:03:00.685936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 15:03:00.685936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:03:00.685936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:03:00.685936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:03:00.685936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:03:00.685936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:03:00.685936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:03:00.686831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:03:00.686831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:03:00.686831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:03:00.686831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:03:00.686831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:03:00.686831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:03:00.686831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:03:00.686831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:03:00.686831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:03:00.686831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:03:00.686831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 15:03:00.686831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:03:00.686831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 15:03:00.686831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:03:00.686831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 649/1290] 10.222.0.44 () {56 vars in 1117 bytes} [Sun Sep 11 15:03:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:03:00.692108 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348efe6d-3822-4635-99d9-9fd3a3b0dbac None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:03:00.800289 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348efe6d-3822-4635-99d9-9fd3a3b0dbac None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:03:00.835241 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd89485-914f-4a4e-b396-e33d22ea3e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:03:00.835861 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd89485-914f-4a4e-b396-e33d22ea3e14 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:00.836114 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd89485-914f-4a4e-b396-e33d22ea3e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:00.836358 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd89485-914f-4a4e-b396-e33d22ea3e14 None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:00.836997 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd89485-914f-4a4e-b396-e33d22ea3e14 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:03:00.837310 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd89485-914f-4a4e-b396-e33d22ea3e14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:03:00.838696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd89485-914f-4a4e-b396-e33d22ea3e14 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:03:00.844449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 642/1291] 10.222.0.44 () {56 vars in 1116 bytes} [Sun Sep 11 15:03:00 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:03:00.850717 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e32680-6e1f-4986-9f82-80da69574ae1 None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:03:00.964647 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e32680-6e1f-4986-9f82-80da69574ae1 None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:03:01.000307 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935976a8-2be1-4d0c-ac74-834840fb307b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:03:01.000926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935976a8-2be1-4d0c-ac74-834840fb307b None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:01.001227 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935976a8-2be1-4d0c-ac74-834840fb307b None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:01.001499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935976a8-2be1-4d0c-ac74-834840fb307b None admin] PATH_INFO: `/v3/services` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:01.002137 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935976a8-2be1-4d0c-ac74-834840fb307b None admin] RBAC: Adding query filter params (type=placement) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:03:01.002474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935976a8-2be1-4d0c-ac74-834840fb307b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:03:01.003839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935976a8-2be1-4d0c-ac74-834840fb307b None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:03:01.008713 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 650/1292] 10.222.0.44 () {56 vars in 1116 bytes} [Sun Sep 11 15:03:00 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:03:03.134766 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac067d84-e5e3-48b5-a76a-d093ad684358 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:03.135363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac067d84-e5e3-48b5-a76a-d093ad684358 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:03.135363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac067d84-e5e3-48b5-a76a-d093ad684358 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:03.136641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 643/1293] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:03:03.142176 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc35655-e3d9-45e3-8aa0-a022a8a5a067 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:03.142634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc35655-e3d9-45e3-8aa0-a022a8a5a067 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:03.142723 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc35655-e3d9-45e3-8aa0-a022a8a5a067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:03.162164 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-8bc35655-e3d9-45e3-8aa0-a022a8a5a067 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:03:03.163364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-8bc35655-e3d9-45e3-8aa0-a022a8a5a067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:03:03.194279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:03:03.235399 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 651/1294] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:03:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:03:03.497087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac067d84-e5e3-48b5-a76a-d093ad684358 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:03:03.531584 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:03:03.605595 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac067d84-e5e3-48b5-a76a-d093ad684358 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:03:03.641504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818eeaac-8b2f-4b4c-888b-54f9993be115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:03:03.642120 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818eeaac-8b2f-4b4c-888b-54f9993be115 None admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:03.642399 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818eeaac-8b2f-4b4c-888b-54f9993be115 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:03.642669 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818eeaac-8b2f-4b4c-888b-54f9993be115 None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:03.643318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818eeaac-8b2f-4b4c-888b-54f9993be115 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:03:03.644703 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818eeaac-8b2f-4b4c-888b-54f9993be115 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:03:03.660597 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 644/1295] 10.222.0.44 () {60 vars in 1141 bytes} [Sun Sep 11 15:03:03 2022] POST /identity/v3/services => generated 246 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:03:05.823680 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779be9b-dd69-4fe4-b3f3-ce81472af10b None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:05.824252 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779be9b-dd69-4fe4-b3f3-ce81472af10b None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:05.824252 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779be9b-dd69-4fe4-b3f3-ce81472af10b None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:05.825591 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 652/1296] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:03:05.831333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43910867-29fe-4669-a86f-9b8c5532fc02 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:05.831819 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43910867-29fe-4669-a86f-9b8c5532fc02 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:05.832024 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43910867-29fe-4669-a86f-9b8c5532fc02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:05.856574 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-43910867-29fe-4669-a86f-9b8c5532fc02 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:03:05.857835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-43910867-29fe-4669-a86f-9b8c5532fc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:03:05.937603 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 645/1297] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:03:05 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:03:06.201947 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779be9b-dd69-4fe4-b3f3-ce81472af10b None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:03:06.308244 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779be9b-dd69-4fe4-b3f3-ce81472af10b None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:03:06.343104 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2662df-7794-47e1-af54-7cbacf61387c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:03:06.343728 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2662df-7794-47e1-af54-7cbacf61387c None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:06.344006 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2662df-7794-47e1-af54-7cbacf61387c None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:06.344267 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2662df-7794-47e1-af54-7cbacf61387c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:06.344826 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2662df-7794-47e1-af54-7cbacf61387c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:03:06.346245 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2662df-7794-47e1-af54-7cbacf61387c None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:03:06.355527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-aa2662df-7794-47e1-af54-7cbacf61387c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 15:03:06.355527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:03:06.355527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 15:03:06.355527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 15:03:06.355527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 15:03:06.355527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 15:03:06.355527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 15:03:06.355527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:03:06.355527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 15:03:06.355527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:03:06.355527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 15:03:06.355527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:03:06.355527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:03:06.355527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:03:06.355527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:03:06.355527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:03:06.355527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:03:06.356681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:03:06.356681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:03:06.356681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:03:06.356681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:03:06.356681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:03:06.356681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:03:06.356681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 15:03:06.356681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 15:03:06.356681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 15:03:06.356681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 15:03:06.356681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:03:06.356681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:03:06.356681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:03:06.356681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:03:06.356681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:03:06.356681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:03:06.357991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:03:06.357991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:03:06.357991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:03:06.357991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:03:06.357991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:03:06.357991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:03:06.357991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:03:06.357991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:03:06.357991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:03:06.357991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:03:06.357991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 15:03:06.357991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:03:06.357991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 15:03:06.357991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:03:06.357991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 653/1298] 10.222.0.44 () {56 vars in 1117 bytes} [Sun Sep 11 15:03:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:03:06.362890 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43910867-29fe-4669-a86f-9b8c5532fc02 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:03:06.382635 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:03:06.467583 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43910867-29fe-4669-a86f-9b8c5532fc02 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:03:06.502315 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743b315-842e-4d31-9725-2eb2a6145d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:03:06.502920 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743b315-842e-4d31-9725-2eb2a6145d93 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:06.503188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743b315-842e-4d31-9725-2eb2a6145d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:06.503451 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743b315-842e-4d31-9725-2eb2a6145d93 None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:06.504072 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6743b315-842e-4d31-9725-2eb2a6145d93 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:03:06.504368 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6743b315-842e-4d31-9725-2eb2a6145d93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:03:06.505884 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6743b315-842e-4d31-9725-2eb2a6145d93 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:03:06.510996 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 646/1299] 10.222.0.44 () {56 vars in 1116 bytes} [Sun Sep 11 15:03:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:03:06.517327 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2662df-7794-47e1-af54-7cbacf61387c None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:03:06.551463 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:03:06.621940 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2662df-7794-47e1-af54-7cbacf61387c None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:03:06.656610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d237621-13af-4d92-b6ba-56af961ae15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:03:06.657243 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d237621-13af-4d92-b6ba-56af961ae15a None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:06.657518 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d237621-13af-4d92-b6ba-56af961ae15a None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:06.657789 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d237621-13af-4d92-b6ba-56af961ae15a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:06.658495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d237621-13af-4d92-b6ba-56af961ae15a None admin] RBAC: Adding query filter params (interface=public, service_id=d03f18378fc440018d26183db0a157c8, region_id=RegionOne) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:03:06.658803 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d237621-13af-4d92-b6ba-56af961ae15a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:03:06.660170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d237621-13af-4d92-b6ba-56af961ae15a None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:03:06.665388 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 654/1300] 10.222.0.44 () {56 vars in 1251 bytes} [Sun Sep 11 15:03:06 2022] GET /identity/v3/endpoints?service_id=d03f18378fc440018d26183db0a157c8&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:03:06.671306 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743b315-842e-4d31-9725-2eb2a6145d93 None admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:03:06.756060 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:03:06.780409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743b315-842e-4d31-9725-2eb2a6145d93 None admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:03:06.816423 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d0878-3b52-4c7a-96c3-35e9a7db6d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:03:06.817031 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d0878-3b52-4c7a-96c3-35e9a7db6d5a None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:06.817339 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d0878-3b52-4c7a-96c3-35e9a7db6d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:06.817608 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d0878-3b52-4c7a-96c3-35e9a7db6d5a None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:06.818209 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d0878-3b52-4c7a-96c3-35e9a7db6d5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:03:06.819577 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d0878-3b52-4c7a-96c3-35e9a7db6d5a None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:03:06.826675 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 647/1301] 10.222.0.44 () {56 vars in 1087 bytes} [Sun Sep 11 15:03:06 2022] GET /identity/v3/services => generated 2191 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:03:08.962920 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb5db6-752f-42f0-a1a6-79f06395a0e0 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:08.963489 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb5db6-752f-42f0-a1a6-79f06395a0e0 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:08.963489 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb5db6-752f-42f0-a1a6-79f06395a0e0 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:08.964802 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 655/1302] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:03:08.970625 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c1d1d-a3a9-4392-bc97-5249171e14fc None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:08.971076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c1d1d-a3a9-4392-bc97-5249171e14fc None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:08.971169 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c1d1d-a3a9-4392-bc97-5249171e14fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:08.990438 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-223c1d1d-a3a9-4392-bc97-5249171e14fc None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:03:08.991828 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-223c1d1d-a3a9-4392-bc97-5249171e14fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:03:09.060278 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 648/1303] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:03:08 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 15:03:09.325890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb5db6-752f-42f0-a1a6-79f06395a0e0 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:03:09.423266 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:03:09.431526 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb5db6-752f-42f0-a1a6-79f06395a0e0 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:03:09.468256 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bf4ba-5844-4595-a6b3-1cd589500cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:03:09.468905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bf4ba-5844-4595-a6b3-1cd589500cdc None admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:09.469180 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bf4ba-5844-4595-a6b3-1cd589500cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:09.469480 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bf4ba-5844-4595-a6b3-1cd589500cdc None admin] PATH_INFO: `/v3/services/placement` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:09.470072 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29bf4ba-5844-4595-a6b3-1cd589500cdc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:03:09.471476 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29bf4ba-5844-4595-a6b3-1cd589500cdc None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:03:09.481598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application [None req-d29bf4ba-5844-4595-a6b3-1cd589500cdc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 15:03:09.481598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:03:09.481598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 15:03:09.481598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 15:03:09.481598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 15:03:09.481598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 15:03:09.481598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 15:03:09.481598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:03:09.481598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 15:03:09.481598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:03:09.481598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 15:03:09.481598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:03:09.481598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 15:03:09.481598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 15:03:09.481598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 15:03:09.481598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 15:03:09.481598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 15:03:09.482815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 15:03:09.482815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 15:03:09.482815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 15:03:09.482815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 15:03:09.482815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 15:03:09.482815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 15:03:09.482815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 15:03:09.482815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 15:03:09.482815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 15:03:09.482815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 15:03:09.482815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 15:03:09.482815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 15:03:09.482815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 15:03:09.482815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 15:03:09.482815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 15:03:09.482815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 15:03:09.483820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 15:03:09.483820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application with Lock( Sep 11 15:03:09.483820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 15:03:09.483820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self._enter() Sep 11 15:03:09.483820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 15:03:09.483820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 15:03:09.483820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 15:03:09.483820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application return self.creator() Sep 11 15:03:09.483820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 15:03:09.483820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application created_value = creator( Sep 11 15:03:09.483820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 15:03:09.483820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 15:03:09.483820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 15:03:09.483820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: ERROR keystone.server.flask.application Sep 11 15:03:09.483820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 656/1304] 10.222.0.44 () {56 vars in 1117 bytes} [Sun Sep 11 15:03:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 15:03:09.489115 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c1d1d-a3a9-4392-bc97-5249171e14fc None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:03:09.568461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:03:09.593536 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c1d1d-a3a9-4392-bc97-5249171e14fc None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:03:09.628900 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786c8b1-9b6f-40d7-b994-bf8ceea7e014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:03:09.629536 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786c8b1-9b6f-40d7-b994-bf8ceea7e014 None admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:09.629809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786c8b1-9b6f-40d7-b994-bf8ceea7e014 None admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:09.630076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786c8b1-9b6f-40d7-b994-bf8ceea7e014 None admin] PATH_INFO: `/v3/services` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:09.630766 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786c8b1-9b6f-40d7-b994-bf8ceea7e014 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:03:09.631068 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786c8b1-9b6f-40d7-b994-bf8ceea7e014 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:03:09.632444 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786c8b1-9b6f-40d7-b994-bf8ceea7e014 None admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:03:09.638127 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 649/1305] 10.222.0.44 () {56 vars in 1116 bytes} [Sun Sep 11 15:03:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:03:09.644279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bf4ba-5844-4595-a6b3-1cd589500cdc None admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:03:09.746536 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bf4ba-5844-4595-a6b3-1cd589500cdc None admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:03:09.781232 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f5ed62-408a-4bf1-971b-a25150297cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:03:09.781841 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5ed62-408a-4bf1-971b-a25150297cd2 None admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:03:09.782112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5ed62-408a-4bf1-971b-a25150297cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:03:09.782381 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5ed62-408a-4bf1-971b-a25150297cd2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:03:09.783028 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f5ed62-408a-4bf1-971b-a25150297cd2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:03:09.784389 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f5ed62-408a-4bf1-971b-a25150297cd2 None admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:03:09.806151 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 657/1306] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:03:09 2022] POST /identity/v3/endpoints => generated 331 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:03:19.634704 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=30 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:03:19.801777 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:04:17.462397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2a2e3-2878-455d-8a5c-47cde5428f5e None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:17.462995 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2a2e3-2878-455d-8a5c-47cde5428f5e None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:17.462995 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2a2e3-2878-455d-8a5c-47cde5428f5e None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:17.464758 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 650/1307] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:04:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:04:17.471977 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab70fd3-cf03-42e3-b22f-4f0aaf6dcf04 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:17.472488 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab70fd3-cf03-42e3-b22f-4f0aaf6dcf04 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:17.472956 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab70fd3-cf03-42e3-b22f-4f0aaf6dcf04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:17.622356 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-3ab70fd3-cf03-42e3-b22f-4f0aaf6dcf04 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:04:17.623722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-3ab70fd3-cf03-42e3-b22f-4f0aaf6dcf04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:17.742641 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 658/1308] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:04:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:04:18.037112 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a5187-d495-4214-8bd8-b05b3bce296d None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:18.037481 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a5187-d495-4214-8bd8-b05b3bce296d None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:18.037753 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a5187-d495-4214-8bd8-b05b3bce296d None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:18.039180 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 651/1309] 10.222.0.44 () {54 vars in 965 bytes} [Sun Sep 11 15:04:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:04:18.045763 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e1289-820b-4744-b313-e8a988f0bfbe None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:18.046048 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e1289-820b-4744-b313-e8a988f0bfbe None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:18.046324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e1289-820b-4744-b313-e8a988f0bfbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:18.146877 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-f84e1289-820b-4744-b313-e8a988f0bfbe None None] MFA Rules not processed for user `697b73f182e6419a84e0c8c6b0fc0c06`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:04:18.148167 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-f84e1289-820b-4744-b313-e8a988f0bfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:18.171136 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:04:18.234965 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 659/1310] 10.222.0.44 () {58 vars in 1062 bytes} [Sun Sep 11 15:04:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:04:18.244574 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555a5187-d495-4214-8bd8-b05b3bce296d None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:04:18.246528 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-555a5187-d495-4214-8bd8-b05b3bce296d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:18.370907 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555a5187-d495-4214-8bd8-b05b3bce296d None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:04:18.449532 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b90bf-99d4-4fc8-adc5-77d9f9d99665 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:04:18.450226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b90bf-99d4-4fc8-adc5-77d9f9d99665 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:18.450438 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b90bf-99d4-4fc8-adc5-77d9f9d99665 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:18.450670 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b90bf-99d4-4fc8-adc5-77d9f9d99665 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:18.452301 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-df7b90bf-99d4-4fc8-adc5-77d9f9d99665 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:18.513452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:04:18.581210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b90bf-99d4-4fc8-adc5-77d9f9d99665 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:04:18.582665 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b90bf-99d4-4fc8-adc5-77d9f9d99665 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:04:18.609042 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 652/1311] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:04:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:04:21.090277 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0114424-d2e9-4c4c-82c4-2ad693d4d396 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:21.090874 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0114424-d2e9-4c4c-82c4-2ad693d4d396 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:21.090874 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0114424-d2e9-4c4c-82c4-2ad693d4d396 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:21.092151 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 660/1312] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:04:21.098198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0146a9-2bcb-4ab6-9c79-f4b85a121007 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:21.098684 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0146a9-2bcb-4ab6-9c79-f4b85a121007 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:21.098802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0146a9-2bcb-4ab6-9c79-f4b85a121007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:21.175804 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-ab0146a9-2bcb-4ab6-9c79-f4b85a121007 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:04:21.177029 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-ab0146a9-2bcb-4ab6-9c79-f4b85a121007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:21.294222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 653/1313] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:04:21.573983 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0114424-d2e9-4c4c-82c4-2ad693d4d396 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:04:21.575773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-a0114424-d2e9-4c4c-82c4-2ad693d4d396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:21.686953 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0114424-d2e9-4c4c-82c4-2ad693d4d396 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:04:21.772157 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:04:21.774873 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc59ed0f-405b-46c5-8111-a32f6ce31ad8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:04:21.775581 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59ed0f-405b-46c5-8111-a32f6ce31ad8 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:21.775975 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59ed0f-405b-46c5-8111-a32f6ce31ad8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:21.776557 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59ed0f-405b-46c5-8111-a32f6ce31ad8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:21.778757 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-fc59ed0f-405b-46c5-8111-a32f6ce31ad8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:21.910751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc59ed0f-405b-46c5-8111-a32f6ce31ad8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:04:21.912112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc59ed0f-405b-46c5-8111-a32f6ce31ad8 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:04:21.932177 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 661/1314] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:04:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:04:24.733185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920e3d2-a2c2-4122-aa32-591b4d3bb8af None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:24.733647 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920e3d2-a2c2-4122-aa32-591b4d3bb8af None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:24.733739 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920e3d2-a2c2-4122-aa32-591b4d3bb8af None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:24.735079 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 654/1315] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:04:24.741094 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214cb9a-bed6-4777-962a-8167e0e8e2ef None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:24.741767 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214cb9a-bed6-4777-962a-8167e0e8e2ef None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:24.741767 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214cb9a-bed6-4777-962a-8167e0e8e2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:24.821255 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-8214cb9a-bed6-4777-962a-8167e0e8e2ef None None] MFA Rules not processed for user `9c5fead1d17444a69ba60c5cf671f4c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:04:24.822626 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-8214cb9a-bed6-4777-962a-8167e0e8e2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:24.880206 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:04:24.929575 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 662/1316] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:04:24 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:04:25.221903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920e3d2-a2c2-4122-aa32-591b4d3bb8af None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:04:25.223986 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-7920e3d2-a2c2-4122-aa32-591b4d3bb8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:25.235892 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:04:25.343898 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920e3d2-a2c2-4122-aa32-591b4d3bb8af None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:04:25.431373 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c4c8a-5acf-46af-9d0c-971498b0ec90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:04:25.432017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c4c8a-5acf-46af-9d0c-971498b0ec90 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:25.432410 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c4c8a-5acf-46af-9d0c-971498b0ec90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:25.432677 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c4c8a-5acf-46af-9d0c-971498b0ec90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:25.434564 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-d38c4c8a-5acf-46af-9d0c-971498b0ec90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:25.507053 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:04:25.558570 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c4c8a-5acf-46af-9d0c-971498b0ec90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:04:25.560069 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c4c8a-5acf-46af-9d0c-971498b0ec90 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:04:25.582614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 655/1317] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:04:25 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:04:28.622049 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e074812-0609-4c35-b8a6-5d11d909d814 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:28.622613 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e074812-0609-4c35-b8a6-5d11d909d814 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:28.622613 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e074812-0609-4c35-b8a6-5d11d909d814 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:28.623800 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 663/1318] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:04:28.630029 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36cb64-e0ab-4eaf-b9e1-879069855f35 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:28.630533 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36cb64-e0ab-4eaf-b9e1-879069855f35 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:28.630848 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36cb64-e0ab-4eaf-b9e1-879069855f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:28.735297 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-7d36cb64-e0ab-4eaf-b9e1-879069855f35 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:04:28.736951 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-7d36cb64-e0ab-4eaf-b9e1-879069855f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:28.879447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 656/1319] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:04:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:04:29.170717 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e074812-0609-4c35-b8a6-5d11d909d814 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:04:29.172384 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-6e074812-0609-4c35-b8a6-5d11d909d814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:29.313741 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e074812-0609-4c35-b8a6-5d11d909d814 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:04:29.406155 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f75b7-81e6-425c-94cb-b9e47eccb705 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:04:29.406661 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f75b7-81e6-425c-94cb-b9e47eccb705 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:29.406935 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f75b7-81e6-425c-94cb-b9e47eccb705 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:29.407257 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f75b7-81e6-425c-94cb-b9e47eccb705 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:29.408921 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-ad0f75b7-81e6-425c-94cb-b9e47eccb705 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:29.416339 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:04:29.540161 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f75b7-81e6-425c-94cb-b9e47eccb705 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:04:29.541751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f75b7-81e6-425c-94cb-b9e47eccb705 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:04:29.561434 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 664/1320] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:04:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:04:32.697909 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a1306-1e46-4826-8e08-0ac08dd6e55d None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:32.698537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a1306-1e46-4826-8e08-0ac08dd6e55d None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:32.698677 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a1306-1e46-4826-8e08-0ac08dd6e55d None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:32.700840 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 657/1321] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:04:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:04:32.707643 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbebe19-1125-468a-a9aa-c391db49c860 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:32.708279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbebe19-1125-468a-a9aa-c391db49c860 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:32.708279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbebe19-1125-468a-a9aa-c391db49c860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:32.784332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-6cbebe19-1125-468a-a9aa-c391db49c860 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:04:32.785995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-6cbebe19-1125-468a-a9aa-c391db49c860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:32.865736 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:04:32.918359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 665/1322] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:04:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:04:33.219747 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a1306-1e46-4826-8e08-0ac08dd6e55d None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:04:33.221642 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-0d6a1306-1e46-4826-8e08-0ac08dd6e55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:33.246538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:04:33.374524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a1306-1e46-4826-8e08-0ac08dd6e55d None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:04:33.475979 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf0c35-d25c-48ac-b2db-978b03d56408 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:04:33.476612 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf0c35-d25c-48ac-b2db-978b03d56408 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:33.476893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf0c35-d25c-48ac-b2db-978b03d56408 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:33.477184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf0c35-d25c-48ac-b2db-978b03d56408 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:33.478885 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-cfbf0c35-d25c-48ac-b2db-978b03d56408 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:33.568693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:04:33.620048 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf0c35-d25c-48ac-b2db-978b03d56408 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:04:33.621778 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf0c35-d25c-48ac-b2db-978b03d56408 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:04:33.644473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 658/1323] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:04:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:04:35.861194 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb8dc3-6b89-4b74-9074-9757dd7d5c88 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:35.861736 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb8dc3-6b89-4b74-9074-9757dd7d5c88 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:35.861827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb8dc3-6b89-4b74-9074-9757dd7d5c88 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:35.863199 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 666/1324] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:04:35.868796 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dd767-de40-4f8f-bb60-37afbe575811 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:35.869268 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dd767-de40-4f8f-bb60-37afbe575811 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:35.869386 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dd767-de40-4f8f-bb60-37afbe575811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:35.938668 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-3a3dd767-de40-4f8f-bb60-37afbe575811 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:04:35.939870 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-3a3dd767-de40-4f8f-bb60-37afbe575811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:36.048454 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 659/1325] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:04:36.337331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb8dc3-6b89-4b74-9074-9757dd7d5c88 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:04:36.339393 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-6dbb8dc3-6b89-4b74-9074-9757dd7d5c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:36.455271 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb8dc3-6b89-4b74-9074-9757dd7d5c88 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:04:36.533201 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a23ff-9fc2-4c67-b7b1-10ce19c2ad13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:04:36.533732 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a23ff-9fc2-4c67-b7b1-10ce19c2ad13 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:36.534011 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a23ff-9fc2-4c67-b7b1-10ce19c2ad13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:36.534328 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a23ff-9fc2-4c67-b7b1-10ce19c2ad13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:36.535976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-130a23ff-9fc2-4c67-b7b1-10ce19c2ad13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:36.554309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:04:36.652802 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a23ff-9fc2-4c67-b7b1-10ce19c2ad13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:04:36.654292 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a23ff-9fc2-4c67-b7b1-10ce19c2ad13 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:04:36.672834 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 667/1326] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:04:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:04:38.930338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ac524-97e8-4948-89f0-b77cb4c7c874 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:38.930911 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ac524-97e8-4948-89f0-b77cb4c7c874 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:38.930911 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ac524-97e8-4948-89f0-b77cb4c7c874 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:38.932137 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 660/1327] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:04:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:04:38.937939 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10268a2f-a226-4d3c-b887-0912387221f7 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:38.938411 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10268a2f-a226-4d3c-b887-0912387221f7 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:38.938492 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10268a2f-a226-4d3c-b887-0912387221f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:39.008047 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-10268a2f-a226-4d3c-b887-0912387221f7 None None] MFA Rules not processed for user `9c5fead1d17444a69ba60c5cf671f4c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:04:39.009364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-10268a2f-a226-4d3c-b887-0912387221f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:39.085761 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:04:39.119527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 668/1328] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:04:38 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:04:39.403104 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ac524-97e8-4948-89f0-b77cb4c7c874 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:04:39.404655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c75ac524-97e8-4948-89f0-b77cb4c7c874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:39.440117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:04:39.521978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ac524-97e8-4948-89f0-b77cb4c7c874 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:04:39.602511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762cd6b9-3ea3-424a-8fed-c1858f7a7790 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:04:39.603119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762cd6b9-3ea3-424a-8fed-c1858f7a7790 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:39.603384 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762cd6b9-3ea3-424a-8fed-c1858f7a7790 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:39.603645 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762cd6b9-3ea3-424a-8fed-c1858f7a7790 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:39.605229 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-762cd6b9-3ea3-424a-8fed-c1858f7a7790 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:39.687551 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:04:39.722036 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762cd6b9-3ea3-424a-8fed-c1858f7a7790 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:04:39.723497 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762cd6b9-3ea3-424a-8fed-c1858f7a7790 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:04:39.742932 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 661/1329] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:04:39 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:04:42.155733 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4a6ec-c261-4ab5-a9f5-8461e74744d0 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:42.156332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4a6ec-c261-4ab5-a9f5-8461e74744d0 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:42.156422 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4a6ec-c261-4ab5-a9f5-8461e74744d0 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:42.157824 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 669/1330] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:04:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:04:42.163717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe9e97-a837-4b9e-af95-9d424e28b953 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:42.164463 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe9e97-a837-4b9e-af95-9d424e28b953 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:42.164555 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe9e97-a837-4b9e-af95-9d424e28b953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:42.237797 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-50fe9e97-a837-4b9e-af95-9d424e28b953 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:04:42.239056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-50fe9e97-a837-4b9e-af95-9d424e28b953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:42.353403 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 662/1331] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:04:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:04:42.646814 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ef8b1-2dc1-4986-808c-3e838077fd86 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:42.647093 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ef8b1-2dc1-4986-808c-3e838077fd86 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:42.647379 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ef8b1-2dc1-4986-808c-3e838077fd86 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:42.648739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 670/1332] 10.222.0.44 () {54 vars in 965 bytes} [Sun Sep 11 15:04:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:04:42.657615 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16226c91-2e64-4fd4-95b2-473fcc503a32 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:42.657896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16226c91-2e64-4fd4-95b2-473fcc503a32 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:42.658167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16226c91-2e64-4fd4-95b2-473fcc503a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:42.706311 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:04:42.741011 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-16226c91-2e64-4fd4-95b2-473fcc503a32 None None] MFA Rules not processed for user `697b73f182e6419a84e0c8c6b0fc0c06`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:04:42.742299 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-16226c91-2e64-4fd4-95b2-473fcc503a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:42.822190 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 663/1333] 10.222.0.44 () {58 vars in 1062 bytes} [Sun Sep 11 15:04:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:04:42.831593 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ef8b1-2dc1-4986-808c-3e838077fd86 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:04:42.833188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-995ef8b1-2dc1-4986-808c-3e838077fd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:42.953614 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ef8b1-2dc1-4986-808c-3e838077fd86 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:04:43.032426 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99358908-ba25-4f72-b517-e7ae35fce514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:04:43.033031 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99358908-ba25-4f72-b517-e7ae35fce514 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:43.033340 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99358908-ba25-4f72-b517-e7ae35fce514 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:43.033605 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99358908-ba25-4f72-b517-e7ae35fce514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:43.035306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-99358908-ba25-4f72-b517-e7ae35fce514 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:43.066181 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:04:43.160550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99358908-ba25-4f72-b517-e7ae35fce514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:04:43.162178 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99358908-ba25-4f72-b517-e7ae35fce514 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:04:43.184200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 671/1334] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:04:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:04:45.636100 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2af05-de79-4a96-b6a2-9c285852aee2 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:45.636689 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2af05-de79-4a96-b6a2-9c285852aee2 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:45.636689 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2af05-de79-4a96-b6a2-9c285852aee2 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:45.637933 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 664/1335] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:04:45.643745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202600f-e0d4-4425-b45c-39bd55d98902 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:45.644397 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202600f-e0d4-4425-b45c-39bd55d98902 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:45.644397 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202600f-e0d4-4425-b45c-39bd55d98902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:45.714401 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-a202600f-e0d4-4425-b45c-39bd55d98902 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:04:45.715646 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-a202600f-e0d4-4425-b45c-39bd55d98902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:45.797023 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:04:45.820456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 672/1336] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:04:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:04:46.104583 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2af05-de79-4a96-b6a2-9c285852aee2 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:04:46.106168 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-70e2af05-de79-4a96-b6a2-9c285852aee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:46.238120 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2af05-de79-4a96-b6a2-9c285852aee2 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:04:46.269119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:04:46.318524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4135dc9c-5c28-45eb-8387-d17cd6b0ea0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:04:46.319162 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135dc9c-5c28-45eb-8387-d17cd6b0ea0b service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:46.319427 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135dc9c-5c28-45eb-8387-d17cd6b0ea0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:46.319698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135dc9c-5c28-45eb-8387-d17cd6b0ea0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:46.321346 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-4135dc9c-5c28-45eb-8387-d17cd6b0ea0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:46.442608 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4135dc9c-5c28-45eb-8387-d17cd6b0ea0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:04:46.443988 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4135dc9c-5c28-45eb-8387-d17cd6b0ea0b service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:04:46.464751 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 665/1337] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:04:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:04:49.457052 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f6aa5-3901-4e05-ac31-502026f285e1 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:49.457661 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f6aa5-3901-4e05-ac31-502026f285e1 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:49.457661 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f6aa5-3901-4e05-ac31-502026f285e1 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:49.458923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 673/1338] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:04:49.464878 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a9819-304a-4957-8d81-0fbca2fb8990 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:49.465383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a9819-304a-4957-8d81-0fbca2fb8990 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:49.465486 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a9819-304a-4957-8d81-0fbca2fb8990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:49.534478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-447a9819-304a-4957-8d81-0fbca2fb8990 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:04:49.535713 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-447a9819-304a-4957-8d81-0fbca2fb8990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:49.544381 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:04:49.642954 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 666/1339] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:04:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:04:49.922589 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41f6aa5-3901-4e05-ac31-502026f285e1 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:04:49.925090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-c41f6aa5-3901-4e05-ac31-502026f285e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:50.049555 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41f6aa5-3901-4e05-ac31-502026f285e1 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:04:50.127925 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f103c1d-d0ac-489a-bd19-091dbcf55170 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:04:50.128543 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f103c1d-d0ac-489a-bd19-091dbcf55170 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:50.128844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f103c1d-d0ac-489a-bd19-091dbcf55170 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:50.129109 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f103c1d-d0ac-489a-bd19-091dbcf55170 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:50.130761 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-8f103c1d-d0ac-489a-bd19-091dbcf55170 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:50.174979 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:04:50.256915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f103c1d-d0ac-489a-bd19-091dbcf55170 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:04:50.258468 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f103c1d-d0ac-489a-bd19-091dbcf55170 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:04:50.285176 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 674/1340] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:04:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:04:54.546055 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b7f27-b4c7-4ee0-8a9e-dc3c11594d4a None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:54.546609 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b7f27-b4c7-4ee0-8a9e-dc3c11594d4a None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:54.546609 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b7f27-b4c7-4ee0-8a9e-dc3c11594d4a None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:54.547843 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 667/1341] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:04:54.553296 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64934390-a02b-4e3d-8c50-9ab7a8ed67c4 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:54.553785 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64934390-a02b-4e3d-8c50-9ab7a8ed67c4 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:54.553785 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64934390-a02b-4e3d-8c50-9ab7a8ed67c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:54.624097 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-64934390-a02b-4e3d-8c50-9ab7a8ed67c4 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:04:54.625334 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-64934390-a02b-4e3d-8c50-9ab7a8ed67c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:54.719316 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:04:54.734494 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 675/1342] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:04:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:04:55.014454 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b7f27-b4c7-4ee0-8a9e-dc3c11594d4a None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:04:55.015995 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-696b7f27-b4c7-4ee0-8a9e-dc3c11594d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:55.126645 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b7f27-b4c7-4ee0-8a9e-dc3c11594d4a None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:04:55.169115 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:04:55.208423 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba00a1-0954-4a7a-bcf5-7b3291efe3ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:04:55.209066 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba00a1-0954-4a7a-bcf5-7b3291efe3ea service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:55.209393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba00a1-0954-4a7a-bcf5-7b3291efe3ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:55.209661 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba00a1-0954-4a7a-bcf5-7b3291efe3ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:55.211296 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-92ba00a1-0954-4a7a-bcf5-7b3291efe3ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:55.340367 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba00a1-0954-4a7a-bcf5-7b3291efe3ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:04:55.341867 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba00a1-0954-4a7a-bcf5-7b3291efe3ea service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:04:55.362640 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 668/1343] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:04:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:04:58.188767 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea7391-3882-4120-8b3e-68ee60bc0fa6 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:58.189407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea7391-3882-4120-8b3e-68ee60bc0fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:58.189407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea7391-3882-4120-8b3e-68ee60bc0fa6 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:58.190810 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 676/1344] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:04:58.196424 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4e719-7d09-4557-b702-1848b9ce5020 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:58.196932 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4e719-7d09-4557-b702-1848b9ce5020 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:58.196932 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4e719-7d09-4557-b702-1848b9ce5020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:58.272731 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-e6d4e719-7d09-4557-b702-1848b9ce5020 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:04:58.274263 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-e6d4e719-7d09-4557-b702-1848b9ce5020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:58.294206 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:04:58.380646 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 669/1345] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:04:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:04:58.657839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea7391-3882-4120-8b3e-68ee60bc0fa6 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:04:58.659482 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-18ea7391-3882-4120-8b3e-68ee60bc0fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:58.779122 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea7391-3882-4120-8b3e-68ee60bc0fa6 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:04:58.885961 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472e443-9a64-4770-894f-50f44b687dd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:04:58.886639 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472e443-9a64-4770-894f-50f44b687dd8 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:04:58.886943 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472e443-9a64-4770-894f-50f44b687dd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:04:58.887306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472e443-9a64-4770-894f-50f44b687dd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:04:58.888982 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-9472e443-9a64-4770-894f-50f44b687dd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:04:58.940693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:04:59.011513 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9472e443-9a64-4770-894f-50f44b687dd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:04:59.012968 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9472e443-9a64-4770-894f-50f44b687dd8 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:04:59.031831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 677/1346] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:04:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:05:03.382610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc412d8-fd86-4033-bbd6-540b7ffffb74 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:03.383137 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc412d8-fd86-4033-bbd6-540b7ffffb74 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:03.383137 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc412d8-fd86-4033-bbd6-540b7ffffb74 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:03.384716 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 670/1347] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:05:03.391117 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cc8d4-095f-4a1a-8bd5-d885d2c5a325 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:03.391907 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cc8d4-095f-4a1a-8bd5-d885d2c5a325 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:03.391907 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cc8d4-095f-4a1a-8bd5-d885d2c5a325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:03.480423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-e17cc8d4-095f-4a1a-8bd5-d885d2c5a325 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:05:03.481606 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-e17cc8d4-095f-4a1a-8bd5-d885d2c5a325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:03.611426 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:03.614315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 678/1348] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:05:03.895900 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc412d8-fd86-4033-bbd6-540b7ffffb74 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:05:03.897631 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-9fc412d8-fd86-4033-bbd6-540b7ffffb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:04.037505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc412d8-fd86-4033-bbd6-540b7ffffb74 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:05:04.087851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:04.114820 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ee5d0-d09c-446e-bff1-194c77db361e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:05:04.115422 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ee5d0-d09c-446e-bff1-194c77db361e service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:04.115687 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ee5d0-d09c-446e-bff1-194c77db361e service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:04.115952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ee5d0-d09c-446e-bff1-194c77db361e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:04.117566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-4f1ee5d0-d09c-446e-bff1-194c77db361e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:04.234109 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ee5d0-d09c-446e-bff1-194c77db361e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:05:04.235859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ee5d0-d09c-446e-bff1-194c77db361e service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:05:04.259540 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 671/1349] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:05:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:05:09.408190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa5b2e-9aab-40ce-86c2-7a1fd3e03311 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:09.408924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa5b2e-9aab-40ce-86c2-7a1fd3e03311 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:09.408924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa5b2e-9aab-40ce-86c2-7a1fd3e03311 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:09.412889 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 679/1350] 10.222.0.44 () {54 vars in 921 bytes} [Sun Sep 11 15:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:05:09.423155 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869393bf-eb27-4ccc-81f9-dc84f60ca1fe None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:09.424206 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869393bf-eb27-4ccc-81f9-dc84f60ca1fe None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:09.424704 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869393bf-eb27-4ccc-81f9-dc84f60ca1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:09.563557 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-869393bf-eb27-4ccc-81f9-dc84f60ca1fe None None] MFA Rules not processed for user `ead06362f8a2444bbfe1405cd602f1c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:05:09.569825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-869393bf-eb27-4ccc-81f9-dc84f60ca1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:09.608290 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:09.779140 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 672/1351] 10.222.0.44 () {58 vars in 1018 bytes} [Sun Sep 11 15:05:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:05:14.206093 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6055a2e-63e2-42a9-b326-eef1a690b797 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:14.207214 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6055a2e-63e2-42a9-b326-eef1a690b797 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:14.207616 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6055a2e-63e2-42a9-b326-eef1a690b797 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:14.209630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 680/1352] 10.222.0.44 () {54 vars in 921 bytes} [Sun Sep 11 15:05:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:05:14.215726 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb06094-17ff-41af-8cf8-4cf3fefc1f3c None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:14.216260 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb06094-17ff-41af-8cf8-4cf3fefc1f3c None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:14.216657 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb06094-17ff-41af-8cf8-4cf3fefc1f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:14.315604 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-1eb06094-17ff-41af-8cf8-4cf3fefc1f3c None None] MFA Rules not processed for user `ead06362f8a2444bbfe1405cd602f1c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:05:14.316828 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-1eb06094-17ff-41af-8cf8-4cf3fefc1f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:14.399373 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:14.441296 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 673/1353] 10.222.0.44 () {58 vars in 1018 bytes} [Sun Sep 11 15:05:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:05:16.041494 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a67749-352f-4688-b4a6-25edec212a35 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:16.042065 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a67749-352f-4688-b4a6-25edec212a35 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:16.042157 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a67749-352f-4688-b4a6-25edec212a35 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:16.043523 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 681/1354] 10.222.0.44 () {54 vars in 921 bytes} [Sun Sep 11 15:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:05:16.048913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88529d0d-a988-454a-bb2a-bfca7b48b754 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:16.049386 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88529d0d-a988-454a-bb2a-bfca7b48b754 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:16.049530 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88529d0d-a988-454a-bb2a-bfca7b48b754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:16.133306 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-88529d0d-a988-454a-bb2a-bfca7b48b754 None None] MFA Rules not processed for user `ead06362f8a2444bbfe1405cd602f1c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:05:16.135942 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-88529d0d-a988-454a-bb2a-bfca7b48b754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:16.262827 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 674/1355] 10.222.0.44 () {58 vars in 1018 bytes} [Sun Sep 11 15:05:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:05:19.949829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481656d0-e652-4b94-a234-418fb70413ac None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:19.950467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481656d0-e652-4b94-a234-418fb70413ac None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:19.950467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481656d0-e652-4b94-a234-418fb70413ac None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:19.951673 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 682/1356] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:05:19.958155 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c71fc5-7436-4430-b45b-51cd20454243 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:19.958975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c71fc5-7436-4430-b45b-51cd20454243 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:19.959154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c71fc5-7436-4430-b45b-51cd20454243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:20.001220 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:20.051374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-94c71fc5-7436-4430-b45b-51cd20454243 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:05:20.052532 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-94c71fc5-7436-4430-b45b-51cd20454243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:20.211400 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 675/1357] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:05:20.568505 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d477d-058d-49c6-8446-e78a067c5b4e None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:20.569256 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d477d-058d-49c6-8446-e78a067c5b4e None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:20.569827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d477d-058d-49c6-8446-e78a067c5b4e None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:20.571713 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 683/1358] 10.222.0.44 () {54 vars in 962 bytes} [Sun Sep 11 15:05:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:05:20.580363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552fe520-ee38-4db7-88bd-460d5a217f46 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:20.580641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552fe520-ee38-4db7-88bd-460d5a217f46 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:20.581060 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552fe520-ee38-4db7-88bd-460d5a217f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:20.680806 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-552fe520-ee38-4db7-88bd-460d5a217f46 None None] MFA Rules not processed for user `453297dac64046d19fc1c331d2d85abc`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:05:20.682169 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-552fe520-ee38-4db7-88bd-460d5a217f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:20.738261 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:20.817412 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 676/1359] 10.222.0.44 () {58 vars in 1059 bytes} [Sun Sep 11 15:05:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:05:20.826316 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d477d-058d-49c6-8446-e78a067c5b4e None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:05:20.829035 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-6e8d477d-058d-49c6-8446-e78a067c5b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:21.074175 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d477d-058d-49c6-8446-e78a067c5b4e None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:05:21.214342 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:21.291083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68927743-aaad-4329-9d2e-ee013d8a3b59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:05:21.292070 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68927743-aaad-4329-9d2e-ee013d8a3b59 service nova] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:21.292401 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68927743-aaad-4329-9d2e-ee013d8a3b59 service nova] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:21.292664 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68927743-aaad-4329-9d2e-ee013d8a3b59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:21.294316 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-68927743-aaad-4329-9d2e-ee013d8a3b59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:21.407397 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68927743-aaad-4329-9d2e-ee013d8a3b59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:05:21.409325 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68927743-aaad-4329-9d2e-ee013d8a3b59 service nova] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:05:21.430704 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 684/1360] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:05:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:05:24.034646 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ec9ee-cc5d-409d-ab08-c17b44e77fd5 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:24.035415 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ec9ee-cc5d-409d-ab08-c17b44e77fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:24.035415 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ec9ee-cc5d-409d-ab08-c17b44e77fd5 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:24.038288 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 677/1361] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:05:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:05:24.042477 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6c884-45f2-4f88-af04-6c7263bcdf0b None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:24.043248 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6c884-45f2-4f88-af04-6c7263bcdf0b None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:24.043248 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6c884-45f2-4f88-af04-6c7263bcdf0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:24.146887 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-d1a6c884-45f2-4f88-af04-6c7263bcdf0b None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:05:24.150274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-d1a6c884-45f2-4f88-af04-6c7263bcdf0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:24.237551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:24.345711 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 685/1362] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:05:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:05:24.639276 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ec9ee-cc5d-409d-ab08-c17b44e77fd5 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:05:24.640846 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-9a4ec9ee-cc5d-409d-ab08-c17b44e77fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:24.857676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ec9ee-cc5d-409d-ab08-c17b44e77fd5 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:05:24.870613 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:25.026120 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec36071-af06-40c4-bc61-5ca15ebad4bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:05:25.027202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec36071-af06-40c4-bc61-5ca15ebad4bb service nova] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:25.027686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec36071-af06-40c4-bc61-5ca15ebad4bb service nova] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:25.028178 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec36071-af06-40c4-bc61-5ca15ebad4bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:25.031049 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-aec36071-af06-40c4-bc61-5ca15ebad4bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:25.148242 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec36071-af06-40c4-bc61-5ca15ebad4bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:05:25.150014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec36071-af06-40c4-bc61-5ca15ebad4bb service nova] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:05:25.173283 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 678/1363] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:05:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:05:28.374922 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76618503-0c57-4eed-bea6-05283c3fa7ca None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:28.375779 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76618503-0c57-4eed-bea6-05283c3fa7ca None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:28.375779 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76618503-0c57-4eed-bea6-05283c3fa7ca None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:28.377922 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 686/1364] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:05:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:05:28.384106 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a039f-99b9-423b-bcae-9aa8fd296dc8 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:28.384928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a039f-99b9-423b-bcae-9aa8fd296dc8 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:28.385070 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a039f-99b9-423b-bcae-9aa8fd296dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:28.403675 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:28.470106 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-818a039f-99b9-423b-bcae-9aa8fd296dc8 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:05:28.472151 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-818a039f-99b9-423b-bcae-9aa8fd296dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:28.599417 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 679/1365] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:05:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:05:28.888854 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b95b3-5b9e-4074-b8f9-6ba687d2286e None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:28.889197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b95b3-5b9e-4074-b8f9-6ba687d2286e None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:28.889472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b95b3-5b9e-4074-b8f9-6ba687d2286e None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:28.891092 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 687/1366] 10.222.0.44 () {54 vars in 962 bytes} [Sun Sep 11 15:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:05:28.898058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a37fc-d034-4f21-8196-1ba724fa3067 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:28.898512 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a37fc-d034-4f21-8196-1ba724fa3067 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:28.898870 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a37fc-d034-4f21-8196-1ba724fa3067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:28.991293 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-8f5a37fc-d034-4f21-8196-1ba724fa3067 None None] MFA Rules not processed for user `453297dac64046d19fc1c331d2d85abc`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:05:28.992695 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-8f5a37fc-d034-4f21-8196-1ba724fa3067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:29.032662 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:29.132433 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 680/1367] 10.222.0.44 () {58 vars in 1059 bytes} [Sun Sep 11 15:05:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:05:29.141421 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b95b3-5b9e-4074-b8f9-6ba687d2286e None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:05:29.143252 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-463b95b3-5b9e-4074-b8f9-6ba687d2286e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:29.317607 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab387848-7b61-4dc5-9db8-f71bdedc2971 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:29.318063 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab387848-7b61-4dc5-9db8-f71bdedc2971 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:29.319104 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab387848-7b61-4dc5-9db8-f71bdedc2971 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:29.321183 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 681/1368] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:05:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:05:29.327572 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d52250-68bc-49a5-a423-07b2206aec30 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:29.327924 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d52250-68bc-49a5-a423-07b2206aec30 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:29.328092 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d52250-68bc-49a5-a423-07b2206aec30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:29.341658 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b95b3-5b9e-4074-b8f9-6ba687d2286e None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:05:29.372822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:29.418822 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-43d52250-68bc-49a5-a423-07b2206aec30 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:05:29.420081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-43d52250-68bc-49a5-a423-07b2206aec30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:29.495046 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6ddf1-c2dc-4fb8-98d6-d3b4d2aa0831 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:05:29.496506 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6ddf1-c2dc-4fb8-98d6-d3b4d2aa0831 service nova] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:29.496955 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6ddf1-c2dc-4fb8-98d6-d3b4d2aa0831 service nova] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:29.497406 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6ddf1-c2dc-4fb8-98d6-d3b4d2aa0831 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:29.499188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-4fc6ddf1-c2dc-4fb8-98d6-d3b4d2aa0831 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:29.550826 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:29.564387 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 682/1369] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:05:29.612859 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc6ddf1-c2dc-4fb8-98d6-d3b4d2aa0831 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:05:29.614507 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc6ddf1-c2dc-4fb8-98d6-d3b4d2aa0831 service nova] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:05:29.634352 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 688/1370] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:05:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:05:29.865292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad41d9d-77d6-4dd5-9090-e37b469304fb None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:29.865725 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad41d9d-77d6-4dd5-9090-e37b469304fb None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:29.865853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad41d9d-77d6-4dd5-9090-e37b469304fb None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:29.867388 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 683/1371] 10.222.0.44 () {54 vars in 964 bytes} [Sun Sep 11 15:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:05:29.873259 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fa9da-58d6-4535-8dae-8cd44f49ce16 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:29.873670 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fa9da-58d6-4535-8dae-8cd44f49ce16 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:29.874155 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fa9da-58d6-4535-8dae-8cd44f49ce16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:29.909197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:29.963355 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-9b1fa9da-58d6-4535-8dae-8cd44f49ce16 None None] MFA Rules not processed for user `8b97e2542db54900a99f6f44b78a5e24`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:05:29.964500 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-9b1fa9da-58d6-4535-8dae-8cd44f49ce16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:30.049522 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 689/1372] 10.222.0.44 () {58 vars in 1061 bytes} [Sun Sep 11 15:05:29 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:05:30.057318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad41d9d-77d6-4dd5-9090-e37b469304fb None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:05:30.058853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-5ad41d9d-77d6-4dd5-9090-e37b469304fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:30.188233 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad41d9d-77d6-4dd5-9090-e37b469304fb None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:05:30.301442 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93193b6-c4ef-43c6-8da0-805b7354d718 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:05:30.302070 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93193b6-c4ef-43c6-8da0-805b7354d718 service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:30.302346 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93193b6-c4ef-43c6-8da0-805b7354d718 service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:30.302598 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93193b6-c4ef-43c6-8da0-805b7354d718 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:30.304533 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b93193b6-c4ef-43c6-8da0-805b7354d718 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:30.368898 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:30.457611 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93193b6-c4ef-43c6-8da0-805b7354d718 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:05:30.459156 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93193b6-c4ef-43c6-8da0-805b7354d718 service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:05:30.485159 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 684/1373] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:05:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:05:31.928629 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7ac2c-8f4c-4bdf-8efe-21b7caae5c12 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:31.929223 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7ac2c-8f4c-4bdf-8efe-21b7caae5c12 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:31.929223 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d7ac2c-8f4c-4bdf-8efe-21b7caae5c12 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:31.930594 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 690/1374] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:05:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:05:31.936457 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0b3b3-77ce-4fcd-af47-714ad1918661 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:31.936955 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0b3b3-77ce-4fcd-af47-714ad1918661 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:31.937068 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0b3b3-77ce-4fcd-af47-714ad1918661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:32.016992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-1aa0b3b3-77ce-4fcd-af47-714ad1918661 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:05:32.018314 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-1aa0b3b3-77ce-4fcd-af47-714ad1918661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:32.136997 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 685/1375] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:05:32.430595 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d7ac2c-8f4c-4bdf-8efe-21b7caae5c12 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:05:32.434305 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-e8d7ac2c-8f4c-4bdf-8efe-21b7caae5c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:32.572605 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:32.621446 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d7ac2c-8f4c-4bdf-8efe-21b7caae5c12 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:05:32.743276 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3c671-4136-4d54-bb13-afbb17a5efc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:05:32.743895 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3c671-4136-4d54-bb13-afbb17a5efc4 service nova] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:32.744173 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3c671-4136-4d54-bb13-afbb17a5efc4 service nova] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:32.744437 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3c671-4136-4d54-bb13-afbb17a5efc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:32.746084 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-3be3c671-4136-4d54-bb13-afbb17a5efc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:32.840846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:32.858409 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be3c671-4136-4d54-bb13-afbb17a5efc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:05:32.860127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be3c671-4136-4d54-bb13-afbb17a5efc4 service nova] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:05:32.879961 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 691/1376] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:05:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:05:35.119928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdf071-edc7-4ffa-ba5d-05744beda5f4 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:35.120462 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdf071-edc7-4ffa-ba5d-05744beda5f4 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:35.120462 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdf071-edc7-4ffa-ba5d-05744beda5f4 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:35.121732 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 686/1377] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:05:35.130797 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3779bc6-5bef-4e08-ac9f-f58b6226a3db None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:35.131315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3779bc6-5bef-4e08-ac9f-f58b6226a3db None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:35.131315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3779bc6-5bef-4e08-ac9f-f58b6226a3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:35.205637 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-a3779bc6-5bef-4e08-ac9f-f58b6226a3db None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:05:35.207020 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-a3779bc6-5bef-4e08-ac9f-f58b6226a3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:35.323992 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 692/1378] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:05:35.623692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdf071-edc7-4ffa-ba5d-05744beda5f4 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:05:35.625204 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-5fbdf071-edc7-4ffa-ba5d-05744beda5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:35.628263 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:35.822428 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdf071-edc7-4ffa-ba5d-05744beda5f4 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:05:35.912114 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:35.957607 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b4ffe-0197-4a64-94f2-dcc3793e51a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:05:35.958594 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b4ffe-0197-4a64-94f2-dcc3793e51a6 service nova] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:35.958981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b4ffe-0197-4a64-94f2-dcc3793e51a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:35.959253 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b4ffe-0197-4a64-94f2-dcc3793e51a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:35.961209 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-db1b4ffe-0197-4a64-94f2-dcc3793e51a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:36.061367 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1b4ffe-0197-4a64-94f2-dcc3793e51a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:05:36.063183 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1b4ffe-0197-4a64-94f2-dcc3793e51a6 service nova] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:05:36.088450 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 687/1379] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:05:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:05:38.403767 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae624f-aa7c-42f4-8048-12200d416fa5 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:38.404340 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae624f-aa7c-42f4-8048-12200d416fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:38.404340 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae624f-aa7c-42f4-8048-12200d416fa5 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:38.406100 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 693/1380] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:05:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:05:38.413516 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ca262-adc1-470c-bb41-89e3fcbcce0c None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:38.414047 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ca262-adc1-470c-bb41-89e3fcbcce0c None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:38.414047 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ca262-adc1-470c-bb41-89e3fcbcce0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:38.504241 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-089ca262-adc1-470c-bb41-89e3fcbcce0c None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:05:38.505614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-089ca262-adc1-470c-bb41-89e3fcbcce0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:38.551601 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:38.618683 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 688/1381] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:05:39.007658 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae624f-aa7c-42f4-8048-12200d416fa5 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:05:39.010010 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-32ae624f-aa7c-42f4-8048-12200d416fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:39.086725 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:39.202503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae624f-aa7c-42f4-8048-12200d416fa5 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:05:39.347624 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15248cce-3d7f-4f9f-9dd3-382132ec990e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:05:39.348179 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15248cce-3d7f-4f9f-9dd3-382132ec990e service nova] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:39.348476 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15248cce-3d7f-4f9f-9dd3-382132ec990e service nova] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:39.348708 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15248cce-3d7f-4f9f-9dd3-382132ec990e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:39.350443 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-15248cce-3d7f-4f9f-9dd3-382132ec990e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:39.374519 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:39.462775 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15248cce-3d7f-4f9f-9dd3-382132ec990e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:05:39.464445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15248cce-3d7f-4f9f-9dd3-382132ec990e service nova] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:05:39.484551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 694/1382] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:05:41.794209 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfb5d7-14f2-43df-afcd-8b3175408f56 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:41.794817 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfb5d7-14f2-43df-afcd-8b3175408f56 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:41.794817 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfb5d7-14f2-43df-afcd-8b3175408f56 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:41.796213 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 689/1383] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:05:41.803540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6eb9b-a618-4827-9669-3f5db3854097 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:41.804051 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6eb9b-a618-4827-9669-3f5db3854097 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:41.804638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6eb9b-a618-4827-9669-3f5db3854097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:41.918370 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-6cc6eb9b-a618-4827-9669-3f5db3854097 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:05:41.920134 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-6cc6eb9b-a618-4827-9669-3f5db3854097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:42.063468 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:42.073990 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 695/1384] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:05:42.384099 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfb5d7-14f2-43df-afcd-8b3175408f56 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:05:42.386092 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-37dfb5d7-14f2-43df-afcd-8b3175408f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:42.598487 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfb5d7-14f2-43df-afcd-8b3175408f56 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:05:42.627077 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:42.755214 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4735e4f-1ff1-4dad-8d3d-1a529a7c8b97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:05:42.755866 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4735e4f-1ff1-4dad-8d3d-1a529a7c8b97 service nova] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:42.756153 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4735e4f-1ff1-4dad-8d3d-1a529a7c8b97 service nova] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:42.756414 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4735e4f-1ff1-4dad-8d3d-1a529a7c8b97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:42.758204 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b4735e4f-1ff1-4dad-8d3d-1a529a7c8b97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:42.869463 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4735e4f-1ff1-4dad-8d3d-1a529a7c8b97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:05:42.871082 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4735e4f-1ff1-4dad-8d3d-1a529a7c8b97 service nova] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:05:42.891235 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 690/1385] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:05:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:05:45.120971 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd83278-713b-469f-acc4-89352658ea2a None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:45.121597 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd83278-713b-469f-acc4-89352658ea2a None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:45.121597 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd83278-713b-469f-acc4-89352658ea2a None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:45.122989 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 696/1386] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:05:45.128235 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a246e35-01be-4cfa-b4ab-a22282ecac79 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:45.128666 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a246e35-01be-4cfa-b4ab-a22282ecac79 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:45.128797 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a246e35-01be-4cfa-b4ab-a22282ecac79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:45.167440 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:45.200964 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-1a246e35-01be-4cfa-b4ab-a22282ecac79 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:05:45.202202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-1a246e35-01be-4cfa-b4ab-a22282ecac79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:45.312319 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 691/1387] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:05:45.605426 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd83278-713b-469f-acc4-89352658ea2a None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:05:45.607017 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-0bd83278-713b-469f-acc4-89352658ea2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:45.776871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd83278-713b-469f-acc4-89352658ea2a None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:05:45.836681 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c92f1d-2192-4c69-a41d-d9667b31864e None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:45.837344 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c92f1d-2192-4c69-a41d-d9667b31864e None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:45.837636 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c92f1d-2192-4c69-a41d-d9667b31864e None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:45.839010 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 692/1388] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 15:05:45.844516 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c4e54-a4ed-47ae-b8ba-2db07a98bbdb None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:45.844787 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c4e54-a4ed-47ae-b8ba-2db07a98bbdb None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:45.845047 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c4e54-a4ed-47ae-b8ba-2db07a98bbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:45.859324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:45.912267 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307fc53-d001-48f0-8c87-6087d4a15d92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:05:45.912885 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307fc53-d001-48f0-8c87-6087d4a15d92 service nova] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:45.914065 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307fc53-d001-48f0-8c87-6087d4a15d92 service nova] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:45.914065 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307fc53-d001-48f0-8c87-6087d4a15d92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:45.915137 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-1307fc53-d001-48f0-8c87-6087d4a15d92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:45.921611 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-280c4e54-a4ed-47ae-b8ba-2db07a98bbdb None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:05:45.922961 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-280c4e54-a4ed-47ae-b8ba-2db07a98bbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:45.985483 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:46.017696 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1307fc53-d001-48f0-8c87-6087d4a15d92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:05:46.019346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1307fc53-d001-48f0-8c87-6087d4a15d92 service nova] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:05:46.035405 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 693/1389] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:05:46.040308 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 697/1390] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:05:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:05:46.340844 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc864b-966e-4d08-a1b3-8c0bfa65020b None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:46.341543 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc864b-966e-4d08-a1b3-8c0bfa65020b None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:46.341543 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc864b-966e-4d08-a1b3-8c0bfa65020b None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:46.342759 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 694/1391] 10.222.0.44 () {54 vars in 966 bytes} [Sun Sep 11 15:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:05:46.348547 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac278dfd-5dae-4816-9b5f-f91a5bda3150 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:46.349021 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac278dfd-5dae-4816-9b5f-f91a5bda3150 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:46.349169 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac278dfd-5dae-4816-9b5f-f91a5bda3150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:46.437823 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-ac278dfd-5dae-4816-9b5f-f91a5bda3150 None None] MFA Rules not processed for user `a1fe5293447346eaafd16818e5bd12f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:05:46.439064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-ac278dfd-5dae-4816-9b5f-f91a5bda3150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:46.463885 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:46.544249 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 698/1392] 10.222.0.44 () {58 vars in 1063 bytes} [Sun Sep 11 15:05:46 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:05:46.562983 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc864b-966e-4d08-a1b3-8c0bfa65020b None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:05:46.564418 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-4dfc864b-966e-4d08-a1b3-8c0bfa65020b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:46.678894 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc864b-966e-4d08-a1b3-8c0bfa65020b None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:05:46.760083 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45188c03-3287-4515-b183-5aa1d3ecf475 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fe5293447346eaafd16818e5bd12f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:05:46.760694 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45188c03-3287-4515-b183-5aa1d3ecf475 service glance] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:46.760965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45188c03-3287-4515-b183-5aa1d3ecf475 service glance] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:46.761349 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45188c03-3287-4515-b183-5aa1d3ecf475 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:46.762952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-45188c03-3287-4515-b183-5aa1d3ecf475 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:46.781930 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:46.907272 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45188c03-3287-4515-b183-5aa1d3ecf475 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:05:46.908666 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45188c03-3287-4515-b183-5aa1d3ecf475 service glance] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:05:46.929597 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 695/1393] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:05:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:05:47.725040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf999a-ece5-4d63-98c9-52e1fbc5b0e6 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:47.725659 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf999a-ece5-4d63-98c9-52e1fbc5b0e6 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:47.725766 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf999a-ece5-4d63-98c9-52e1fbc5b0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:47.797429 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-7edf999a-ece5-4d63-98c9-52e1fbc5b0e6 None None] MFA Rules not processed for user `3fada78ceaeb4f3bad77c516f6dfcfd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:05:47.798688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-7edf999a-ece5-4d63-98c9-52e1fbc5b0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:47.895713 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 699/1394] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 15:05:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:05:47.943012 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbc02b-100d-4e6e-81ae-482de0c8d2db None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:47.943581 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbc02b-100d-4e6e-81ae-482de0c8d2db None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:47.943581 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbc02b-100d-4e6e-81ae-482de0c8d2db None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:47.945177 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 696/1395] 10.222.0.44 () {54 vars in 943 bytes} [Sun Sep 11 15:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:05:47.951918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d7edb-74d6-4738-adf1-cf0e9ac09f2b None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:47.952198 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d7edb-74d6-4738-adf1-cf0e9ac09f2b None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:47.952462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d7edb-74d6-4738-adf1-cf0e9ac09f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:48.007264 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:48.044686 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-255d7edb-74d6-4738-adf1-cf0e9ac09f2b None None] MFA Rules not processed for user `05aa5e40ee4542719657bc03b2015e72`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:05:48.045942 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-255d7edb-74d6-4738-adf1-cf0e9ac09f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:48.161816 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 700/1396] 10.222.0.44 () {58 vars in 1040 bytes} [Sun Sep 11 15:05:47 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:05:48.171118 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffbc02b-100d-4e6e-81ae-482de0c8d2db None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:05:48.172645 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-dffbc02b-100d-4e6e-81ae-482de0c8d2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:48.303686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:48.313524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f2a92-5489-446f-a8a7-52808b7d0ed5 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:48.314113 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f2a92-5489-446f-a8a7-52808b7d0ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:48.314618 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f2a92-5489-446f-a8a7-52808b7d0ed5 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:48.316858 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 701/1397] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:05:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 15:05:48.323383 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09866916-3dae-405c-9112-e29602670729 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:48.323747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09866916-3dae-405c-9112-e29602670729 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:48.324099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09866916-3dae-405c-9112-e29602670729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:48.335138 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffbc02b-100d-4e6e-81ae-482de0c8d2db None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:05:48.401907 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-09866916-3dae-405c-9112-e29602670729 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:05:48.403121 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-09866916-3dae-405c-9112-e29602670729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:48.452348 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:48.455137 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce86ae-9b34-4e64-b877-a6acedac9a0d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05aa5e40ee4542719657bc03b2015e72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:05:48.455765 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce86ae-9b34-4e64-b877-a6acedac9a0d service swift] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:48.456041 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce86ae-9b34-4e64-b877-a6acedac9a0d service swift] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:48.456336 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce86ae-9b34-4e64-b877-a6acedac9a0d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:48.458045 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-7cce86ae-9b34-4e64-b877-a6acedac9a0d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:48.514807 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 702/1398] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:05:48.556247 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce86ae-9b34-4e64-b877-a6acedac9a0d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:05:48.557963 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce86ae-9b34-4e64-b877-a6acedac9a0d service swift] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:05:48.567431 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 697/1399] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 15:05:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 15:05:48.801794 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09866916-3dae-405c-9112-e29602670729 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:05:48.803170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-09866916-3dae-405c-9112-e29602670729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:48.965505 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09866916-3dae-405c-9112-e29602670729 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:05:49.051080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:49.130772 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df0119-d399-4c2d-98a9-12f3c9e1e000 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:05:49.131406 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df0119-d399-4c2d-98a9-12f3c9e1e000 service nova] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:49.131684 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df0119-d399-4c2d-98a9-12f3c9e1e000 service nova] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:49.133264 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df0119-d399-4c2d-98a9-12f3c9e1e000 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:49.133799 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-a4df0119-d399-4c2d-98a9-12f3c9e1e000 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:49.291748 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df0119-d399-4c2d-98a9-12f3c9e1e000 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:05:49.293559 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df0119-d399-4c2d-98a9-12f3c9e1e000 service nova] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:05:49.314239 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 703/1400] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:05:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:05:51.579092 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f35751-0549-4475-b819-dfaeed855858 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:51.580018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f35751-0549-4475-b819-dfaeed855858 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:51.580289 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f35751-0549-4475-b819-dfaeed855858 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:51.582052 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 698/1401] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:05:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:05:51.587599 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69395735-c0a1-4f5f-92b7-d84be8a4e1ee None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:51.588116 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69395735-c0a1-4f5f-92b7-d84be8a4e1ee None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:51.588116 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69395735-c0a1-4f5f-92b7-d84be8a4e1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:51.668474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-69395735-c0a1-4f5f-92b7-d84be8a4e1ee None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:05:51.670015 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-69395735-c0a1-4f5f-92b7-d84be8a4e1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:51.693431 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:51.780399 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 704/1402] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:05:52.069237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f35751-0549-4475-b819-dfaeed855858 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:05:52.070993 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-71f35751-0549-4475-b819-dfaeed855858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:52.085799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:52.251241 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f35751-0549-4475-b819-dfaeed855858 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:05:52.349296 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:52.378280 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efde2f2-b4a7-4f69-8445-12157df21aca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:05:52.378894 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efde2f2-b4a7-4f69-8445-12157df21aca service nova] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:52.379165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efde2f2-b4a7-4f69-8445-12157df21aca service nova] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:52.379432 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efde2f2-b4a7-4f69-8445-12157df21aca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:52.381055 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-2efde2f2-b4a7-4f69-8445-12157df21aca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:52.499944 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efde2f2-b4a7-4f69-8445-12157df21aca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:05:52.502095 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efde2f2-b4a7-4f69-8445-12157df21aca service nova] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:05:52.524670 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 699/1403] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:05:54.791695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6b857-1b95-4316-9b43-f932d7b08ed5 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:54.792179 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6b857-1b95-4316-9b43-f932d7b08ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:54.792290 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6b857-1b95-4316-9b43-f932d7b08ed5 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:54.793752 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 705/1404] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:05:54.799051 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c9e82-82df-4d78-a029-271270e6b8c3 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:54.799545 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c9e82-82df-4d78-a029-271270e6b8c3 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:54.799647 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c9e82-82df-4d78-a029-271270e6b8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:54.877173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-df1c9e82-82df-4d78-a029-271270e6b8c3 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:05:54.877663 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-df1c9e82-82df-4d78-a029-271270e6b8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:54.983418 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:55.055112 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 700/1405] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:05:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:05:55.346067 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6b857-1b95-4316-9b43-f932d7b08ed5 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:05:55.347582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-f2c6b857-1b95-4316-9b43-f932d7b08ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:55.519038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6b857-1b95-4316-9b43-f932d7b08ed5 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:05:55.522826 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:55.650663 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0660955-fd56-493e-96a8-d897e318e93b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:05:55.651188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0660955-fd56-493e-96a8-d897e318e93b service nova] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:55.651460 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0660955-fd56-493e-96a8-d897e318e93b service nova] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:55.651727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0660955-fd56-493e-96a8-d897e318e93b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:55.653386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-a0660955-fd56-493e-96a8-d897e318e93b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:55.764324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0660955-fd56-493e-96a8-d897e318e93b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:05:55.767195 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0660955-fd56-493e-96a8-d897e318e93b service nova] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:05:55.788217 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 706/1406] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:05:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:05:58.120655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a4d1d-1d43-471d-827d-dc55ee83b675 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:58.121273 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a4d1d-1d43-471d-827d-dc55ee83b675 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:58.121492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a4d1d-1d43-471d-827d-dc55ee83b675 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:58.123297 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 701/1407] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:05:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:05:58.128900 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f3cc3-f68e-4755-a925-6ec793e31736 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:58.129414 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f3cc3-f68e-4755-a925-6ec793e31736 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:58.129529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f3cc3-f68e-4755-a925-6ec793e31736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:58.141561 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:58.204717 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-3f3f3cc3-f68e-4755-a925-6ec793e31736 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:05:58.205968 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-3f3f3cc3-f68e-4755-a925-6ec793e31736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:58.326573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 707/1408] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:05:58.613963 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a4d1d-1d43-471d-827d-dc55ee83b675 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:05:58.615503 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-662a4d1d-1d43-471d-827d-dc55ee83b675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:58.782154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a4d1d-1d43-471d-827d-dc55ee83b675 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:05:58.826230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:05:58.920152 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa508e-b17d-4393-b06d-719680134414 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:05:58.920679 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa508e-b17d-4393-b06d-719680134414 service nova] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:05:58.920947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa508e-b17d-4393-b06d-719680134414 service nova] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:05:58.921242 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa508e-b17d-4393-b06d-719680134414 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:05:58.922938 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-e7fa508e-b17d-4393-b06d-719680134414 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:05:59.031160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fa508e-b17d-4393-b06d-719680134414 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:05:59.033273 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fa508e-b17d-4393-b06d-719680134414 service nova] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:05:59.054248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 702/1409] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:05:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:06:08.324947 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=61 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:09.051687 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=83 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:10.982978 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303b9f8-3cee-4f37-aacd-782284039927 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:10.983608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303b9f8-3cee-4f37-aacd-782284039927 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:10.983608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303b9f8-3cee-4f37-aacd-782284039927 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:10.984985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 708/1410] 10.222.0.44 () {54 vars in 918 bytes} [Sun Sep 11 15:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 15:06:10.991330 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5f234-0ec2-4b0f-a01b-7e4227ded656 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:10.992201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5f234-0ec2-4b0f-a01b-7e4227ded656 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:10.992201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5f234-0ec2-4b0f-a01b-7e4227ded656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:11.082741 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-95d5f234-0ec2-4b0f-a01b-7e4227ded656 None None] MFA Rules not processed for user `ead06362f8a2444bbfe1405cd602f1c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:06:11.083900 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-95d5f234-0ec2-4b0f-a01b-7e4227ded656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:11.199566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 703/1411] 10.222.0.44 () {58 vars in 1015 bytes} [Sun Sep 11 15:06:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:06:11.215044 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1c6bb-f7dd-453f-af42-5a9629dba1ac None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:11.215321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1c6bb-f7dd-453f-af42-5a9629dba1ac None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:11.215589 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1c6bb-f7dd-453f-af42-5a9629dba1ac None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:11.216942 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 709/1412] 10.222.0.44 () {54 vars in 918 bytes} [Sun Sep 11 15:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:06:11.222904 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad76e2d-eb09-4a90-acea-d4477e175c71 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:11.223153 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad76e2d-eb09-4a90-acea-d4477e175c71 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:11.223443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad76e2d-eb09-4a90-acea-d4477e175c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:11.308656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:11.312543 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-2ad76e2d-eb09-4a90-acea-d4477e175c71 None None] MFA Rules not processed for user `453297dac64046d19fc1c331d2d85abc`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:06:11.314132 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-2ad76e2d-eb09-4a90-acea-d4477e175c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:11.452207 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 704/1413] 10.222.0.44 () {58 vars in 1015 bytes} [Sun Sep 11 15:06:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:06:14.071553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85edd8bc-8883-4748-966c-5e317330c6ce None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:14.072148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85edd8bc-8883-4748-966c-5e317330c6ce None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:14.072148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85edd8bc-8883-4748-966c-5e317330c6ce None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:14.073459 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 15:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:06:14.081391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8eb94-b31a-45fb-9c0b-cccd77ebab1b None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:14.081918 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8eb94-b31a-45fb-9c0b-cccd77ebab1b None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:14.081918 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8eb94-b31a-45fb-9c0b-cccd77ebab1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:14.156228 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-d9e8eb94-b31a-45fb-9c0b-cccd77ebab1b None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:06:14.157460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-d9e8eb94-b31a-45fb-9c0b-cccd77ebab1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:14.226882 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:14.280242 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 15:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:06:14.567356 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85edd8bc-8883-4748-966c-5e317330c6ce None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:06:14.568910 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-85edd8bc-8883-4748-966c-5e317330c6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:14.738476 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85edd8bc-8883-4748-966c-5e317330c6ce None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:06:14.820780 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:14.865224 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6689542-0f1f-4404-b1d8-a0fc55bc17a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:06:14.865826 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6689542-0f1f-4404-b1d8-a0fc55bc17a4 service nova] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:14.866091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6689542-0f1f-4404-b1d8-a0fc55bc17a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:14.866352 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6689542-0f1f-4404-b1d8-a0fc55bc17a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:14.867968 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-d6689542-0f1f-4404-b1d8-a0fc55bc17a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:14.975357 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6689542-0f1f-4404-b1d8-a0fc55bc17a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:06:14.976995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6689542-0f1f-4404-b1d8-a0fc55bc17a4 service nova] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:06:14.998381 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 711/1416] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:06:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:06:16.216516 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7360d-ce96-4171-a764-6528dbe5d3a0 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:16.217078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7360d-ce96-4171-a764-6528dbe5d3a0 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:16.217078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7360d-ce96-4171-a764-6528dbe5d3a0 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:16.218485 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 15:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:06:16.225780 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a9a9a-1ec0-4146-9045-756db359739a None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:16.226370 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a9a9a-1ec0-4146-9045-756db359739a None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:16.226370 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a9a9a-1ec0-4146-9045-756db359739a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:16.313009 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-563a9a9a-1ec0-4146-9045-756db359739a None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:06:16.314284 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-563a9a9a-1ec0-4146-9045-756db359739a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:16.347198 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:16.420546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 15:06:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:06:16.572151 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7360d-ce96-4171-a764-6528dbe5d3a0 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:06:16.573711 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-2ad7360d-ce96-4171-a764-6528dbe5d3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:16.738410 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7360d-ce96-4171-a764-6528dbe5d3a0 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:06:16.785904 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:16.863802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55e4b9-8be0-425e-b718-4b363ba0783d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:06:16.864309 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55e4b9-8be0-425e-b718-4b363ba0783d service nova] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:16.864576 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55e4b9-8be0-425e-b718-4b363ba0783d service nova] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:16.864837 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55e4b9-8be0-425e-b718-4b363ba0783d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:16.866471 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-4d55e4b9-8be0-425e-b718-4b363ba0783d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:16.966273 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d55e4b9-8be0-425e-b718-4b363ba0783d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:06:16.967885 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d55e4b9-8be0-425e-b718-4b363ba0783d service nova] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:06:16.986566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 707/1419] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:06:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:06:18.137301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4522ac9b-0367-47e5-9a18-30ee34f0c248 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:18.137997 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4522ac9b-0367-47e5-9a18-30ee34f0c248 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:18.137997 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4522ac9b-0367-47e5-9a18-30ee34f0c248 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:18.139215 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 15:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:06:18.146598 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba2088-f927-4407-8844-6bf06938d82c None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:18.147219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba2088-f927-4407-8844-6bf06938d82c None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:18.147219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba2088-f927-4407-8844-6bf06938d82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:18.209074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:18.236497 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-dbba2088-f927-4407-8844-6bf06938d82c None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:06:18.237896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-dbba2088-f927-4407-8844-6bf06938d82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:18.364953 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 15:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:06:18.520258 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4522ac9b-0367-47e5-9a18-30ee34f0c248 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:06:18.521912 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-4522ac9b-0367-47e5-9a18-30ee34f0c248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:18.713240 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4522ac9b-0367-47e5-9a18-30ee34f0c248 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:06:18.727001 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:18.855214 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da81b29-37a1-4059-957c-848dc3ca7cb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:06:18.855835 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da81b29-37a1-4059-957c-848dc3ca7cb3 service nova] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:18.856108 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da81b29-37a1-4059-957c-848dc3ca7cb3 service nova] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:18.856396 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da81b29-37a1-4059-957c-848dc3ca7cb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:18.858243 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-8da81b29-37a1-4059-957c-848dc3ca7cb3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:18.960000 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da81b29-37a1-4059-957c-848dc3ca7cb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:06:18.962082 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da81b29-37a1-4059-957c-848dc3ca7cb3 service nova] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:06:18.982342 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 714/1422] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:06:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:06:20.184008 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154bd5a-5d77-4dcb-ad90-55f0c2d63478 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:20.184858 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154bd5a-5d77-4dcb-ad90-55f0c2d63478 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:20.184858 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154bd5a-5d77-4dcb-ad90-55f0c2d63478 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:20.186577 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 15:06:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:06:20.194152 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c3e2b-1fe8-45fc-8792-a681261937c6 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:20.194910 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c3e2b-1fe8-45fc-8792-a681261937c6 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:20.194910 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c3e2b-1fe8-45fc-8792-a681261937c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:20.226384 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:20.308765 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-a48c3e2b-1fe8-45fc-8792-a681261937c6 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:06:20.310341 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-a48c3e2b-1fe8-45fc-8792-a681261937c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:20.414488 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 15:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:06:20.572240 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154bd5a-5d77-4dcb-ad90-55f0c2d63478 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:06:20.575006 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c154bd5a-5d77-4dcb-ad90-55f0c2d63478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:20.751804 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:20.794090 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154bd5a-5d77-4dcb-ad90-55f0c2d63478 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:06:20.930105 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a6b1f-2ac9-45de-ab80-ae77a2adb601 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:06:20.930689 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a6b1f-2ac9-45de-ab80-ae77a2adb601 service nova] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:20.931014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a6b1f-2ac9-45de-ab80-ae77a2adb601 service nova] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:20.931325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a6b1f-2ac9-45de-ab80-ae77a2adb601 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:20.933042 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-783a6b1f-2ac9-45de-ab80-ae77a2adb601 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:21.020903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:21.036726 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783a6b1f-2ac9-45de-ab80-ae77a2adb601 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:06:21.038421 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783a6b1f-2ac9-45de-ab80-ae77a2adb601 service nova] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:06:21.059766 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 710/1425] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:06:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:06:22.210309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4dea91-0242-4c7f-bb23-d96cf76d2770 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:22.211582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4dea91-0242-4c7f-bb23-d96cf76d2770 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:22.211582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4dea91-0242-4c7f-bb23-d96cf76d2770 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:22.212579 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 15:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:06:22.220008 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc91ef-8cae-431b-ac93-bcba76e38c8c None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:22.220880 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc91ef-8cae-431b-ac93-bcba76e38c8c None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:22.220880 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc91ef-8cae-431b-ac93-bcba76e38c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:22.315037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-c3dc91ef-8cae-431b-ac93-bcba76e38c8c None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:06:22.316381 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c3dc91ef-8cae-431b-ac93-bcba76e38c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:22.428186 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 15:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:06:22.580081 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4dea91-0242-4c7f-bb23-d96cf76d2770 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:06:22.581627 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-8d4dea91-0242-4c7f-bb23-d96cf76d2770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:22.708363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:22.781191 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4dea91-0242-4c7f-bb23-d96cf76d2770 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:06:22.905346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e2730-1d4b-4f93-b477-bb985ab854dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:06:22.905880 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e2730-1d4b-4f93-b477-bb985ab854dd service nova] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:22.906220 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e2730-1d4b-4f93-b477-bb985ab854dd service nova] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:22.906490 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e2730-1d4b-4f93-b477-bb985ab854dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:22.908179 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-b98e2730-1d4b-4f93-b477-bb985ab854dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:22.962420 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:23.006976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e2730-1d4b-4f93-b477-bb985ab854dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:06:23.008616 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e2730-1d4b-4f93-b477-bb985ab854dd service nova] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:06:23.028649 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 717/1428] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:06:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:06:24.108958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc40cc6-a948-41fb-abc3-5e1153b69f99 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:24.109771 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc40cc6-a948-41fb-abc3-5e1153b69f99 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:24.110103 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc40cc6-a948-41fb-abc3-5e1153b69f99 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:24.111622 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 15:06:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:06:24.119750 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867abab9-a7f0-4166-b8c0-4313a1cecc74 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:24.120343 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867abab9-a7f0-4166-b8c0-4313a1cecc74 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:24.120343 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867abab9-a7f0-4166-b8c0-4313a1cecc74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:24.211419 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-867abab9-a7f0-4166-b8c0-4313a1cecc74 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:06:24.212667 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-867abab9-a7f0-4166-b8c0-4313a1cecc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:24.322241 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 15:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:06:24.474185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc40cc6-a948-41fb-abc3-5e1153b69f99 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:06:24.475639 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-cbc40cc6-a948-41fb-abc3-5e1153b69f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:24.532481 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:24.642704 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc40cc6-a948-41fb-abc3-5e1153b69f99 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:06:24.763407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39d0ae-00af-4f0c-b9ce-0b4f185d68c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:06:24.764060 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39d0ae-00af-4f0c-b9ce-0b4f185d68c6 service nova] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:24.764337 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39d0ae-00af-4f0c-b9ce-0b4f185d68c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:24.764603 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39d0ae-00af-4f0c-b9ce-0b4f185d68c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:24.766305 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-cf39d0ae-00af-4f0c-b9ce-0b4f185d68c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:24.774044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:24.862724 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39d0ae-00af-4f0c-b9ce-0b4f185d68c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:06:24.864374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39d0ae-00af-4f0c-b9ce-0b4f185d68c6 service nova] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:06:24.882956 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 713/1431] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:06:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:06:25.971178 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d8c1e-08d5-4219-8de5-bfbb30f7c06e None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:25.971827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d8c1e-08d5-4219-8de5-bfbb30f7c06e None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:25.971827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d8c1e-08d5-4219-8de5-bfbb30f7c06e None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:25.973067 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 15:06:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:06:25.981441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c0bda-fdca-4db3-85cb-37b06f38af89 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:25.982052 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c0bda-fdca-4db3-85cb-37b06f38af89 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:25.982311 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c0bda-fdca-4db3-85cb-37b06f38af89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:26.093046 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-0d3c0bda-fdca-4db3-85cb-37b06f38af89 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:06:26.094681 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-0d3c0bda-fdca-4db3-85cb-37b06f38af89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:26.170656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:26.200699 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 15:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:06:26.350126 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d8c1e-08d5-4219-8de5-bfbb30f7c06e None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:06:26.351687 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-3f3d8c1e-08d5-4219-8de5-bfbb30f7c06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:26.372616 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:26.513909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d8c1e-08d5-4219-8de5-bfbb30f7c06e None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:06:26.605296 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:26.633000 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca8269-3c9b-45f1-8977-fe2e4c162bb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:06:26.633642 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca8269-3c9b-45f1-8977-fe2e4c162bb8 service nova] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:26.633882 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca8269-3c9b-45f1-8977-fe2e4c162bb8 service nova] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:26.634142 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca8269-3c9b-45f1-8977-fe2e4c162bb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:26.635783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-c8ca8269-3c9b-45f1-8977-fe2e4c162bb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:26.739312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca8269-3c9b-45f1-8977-fe2e4c162bb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:06:26.740996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca8269-3c9b-45f1-8977-fe2e4c162bb8 service nova] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:06:26.763955 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 720/1434] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:06:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:06:27.835338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b09fcd-95b6-4dca-a8f5-ab857de8d949 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:27.836347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b09fcd-95b6-4dca-a8f5-ab857de8d949 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:27.836347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b09fcd-95b6-4dca-a8f5-ab857de8d949 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:27.839801 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 15:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:06:27.845208 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289fd07-24f6-4141-9de5-5ca4a5fcc3f0 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:27.845765 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289fd07-24f6-4141-9de5-5ca4a5fcc3f0 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:27.845765 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289fd07-24f6-4141-9de5-5ca4a5fcc3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:27.929890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-c289fd07-24f6-4141-9de5-5ca4a5fcc3f0 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:06:27.931193 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-c289fd07-24f6-4141-9de5-5ca4a5fcc3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:27.981970 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:28.045577 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 15:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:06:28.194891 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b09fcd-95b6-4dca-a8f5-ab857de8d949 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:06:28.196393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b9b09fcd-95b6-4dca-a8f5-ab857de8d949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:28.362960 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b09fcd-95b6-4dca-a8f5-ab857de8d949 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:06:28.422127 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:28.485093 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c8118-1f9f-4b4f-8a64-990c2b117a35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:06:28.485630 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c8118-1f9f-4b4f-8a64-990c2b117a35 service nova] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:28.485900 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c8118-1f9f-4b4f-8a64-990c2b117a35 service nova] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:28.486140 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c8118-1f9f-4b4f-8a64-990c2b117a35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:28.487750 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c57c8118-1f9f-4b4f-8a64-990c2b117a35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:28.577624 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c8118-1f9f-4b4f-8a64-990c2b117a35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:06:28.579252 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c8118-1f9f-4b4f-8a64-990c2b117a35 service nova] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:06:28.597757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 716/1437] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:06:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:06:29.663570 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee789c-4036-4845-a981-6773afea43bf None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:29.664170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee789c-4036-4845-a981-6773afea43bf None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:29.664170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee789c-4036-4845-a981-6773afea43bf None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:29.665706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 15:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:06:29.674793 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7cf0b9-3b8c-4a5e-8795-d3d7eb53515c None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:29.675330 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7cf0b9-3b8c-4a5e-8795-d3d7eb53515c None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:29.675435 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7cf0b9-3b8c-4a5e-8795-d3d7eb53515c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:29.759366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-1d7cf0b9-3b8c-4a5e-8795-d3d7eb53515c None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:06:29.760604 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-1d7cf0b9-3b8c-4a5e-8795-d3d7eb53515c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:29.763033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:29.873284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 15:06:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:06:30.024500 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee789c-4036-4845-a981-6773afea43bf None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:06:30.026336 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-ecee789c-4036-4845-a981-6773afea43bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:30.200090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee789c-4036-4845-a981-6773afea43bf None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:06:30.243500 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:30.344601 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a458e3-3206-4938-9246-6b7625fb60f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:06:30.345269 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a458e3-3206-4938-9246-6b7625fb60f8 service nova] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:30.345542 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a458e3-3206-4938-9246-6b7625fb60f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:30.345813 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a458e3-3206-4938-9246-6b7625fb60f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:30.347585 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-33a458e3-3206-4938-9246-6b7625fb60f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:30.458965 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a458e3-3206-4938-9246-6b7625fb60f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:06:30.460946 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a458e3-3206-4938-9246-6b7625fb60f8 service nova] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:06:30.480266 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 723/1440] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:06:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:06:32.569617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf0671-ba54-4b92-974c-66425dbda835 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:32.570151 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf0671-ba54-4b92-974c-66425dbda835 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:32.570151 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf0671-ba54-4b92-974c-66425dbda835 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:32.571467 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 15:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:06:32.577278 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd08d76-e555-4c0a-b4d9-5963e0111302 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:32.577885 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd08d76-e555-4c0a-b4d9-5963e0111302 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:32.577885 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd08d76-e555-4c0a-b4d9-5963e0111302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:32.621245 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:32.649920 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-8fd08d76-e555-4c0a-b4d9-5963e0111302 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:06:32.651157 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-8fd08d76-e555-4c0a-b4d9-5963e0111302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:32.757620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 15:06:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:06:33.041631 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf0671-ba54-4b92-974c-66425dbda835 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:06:33.043313 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-1ddf0671-ba54-4b92-974c-66425dbda835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:33.155209 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf0671-ba54-4b92-974c-66425dbda835 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:06:33.190577 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:33.252789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920d352-934a-4328-8d9c-cd30ad19ef9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:06:33.253861 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920d352-934a-4328-8d9c-cd30ad19ef9e service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:33.254355 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920d352-934a-4328-8d9c-cd30ad19ef9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:33.254845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920d352-934a-4328-8d9c-cd30ad19ef9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:33.258023 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-2920d352-934a-4328-8d9c-cd30ad19ef9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:33.452183 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2920d352-934a-4328-8d9c-cd30ad19ef9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:06:33.453638 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2920d352-934a-4328-8d9c-cd30ad19ef9e service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:06:33.475896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 719/1443] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:06:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:06:35.832508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5878a6-f0dc-40c5-90fd-5a4658dc4dd8 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:35.833100 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5878a6-f0dc-40c5-90fd-5a4658dc4dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:35.833100 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5878a6-f0dc-40c5-90fd-5a4658dc4dd8 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:35.834510 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 15:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:06:35.841103 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd240e-3744-4c14-8b66-e1ddcb2a820d None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:35.841621 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd240e-3744-4c14-8b66-e1ddcb2a820d None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:35.841729 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd240e-3744-4c14-8b66-e1ddcb2a820d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:35.914349 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-8fcd240e-3744-4c14-8b66-e1ddcb2a820d None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:06:35.915557 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-8fcd240e-3744-4c14-8b66-e1ddcb2a820d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:35.929675 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:36.019495 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 15:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:06:36.301349 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5878a6-f0dc-40c5-90fd-5a4658dc4dd8 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:06:36.302923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-bf5878a6-f0dc-40c5-90fd-5a4658dc4dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:36.417461 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5878a6-f0dc-40c5-90fd-5a4658dc4dd8 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:06:36.432066 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:36.500751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517105a3-f8ea-4437-a645-c06d3bbac045 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:06:36.501411 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517105a3-f8ea-4437-a645-c06d3bbac045 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:36.501683 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517105a3-f8ea-4437-a645-c06d3bbac045 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:36.501946 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517105a3-f8ea-4437-a645-c06d3bbac045 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:36.503605 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-517105a3-f8ea-4437-a645-c06d3bbac045 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:36.624944 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517105a3-f8ea-4437-a645-c06d3bbac045 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:06:36.626349 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517105a3-f8ea-4437-a645-c06d3bbac045 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:06:36.645658 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 726/1446] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:06:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:06:39.186580 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10dddbe-cebd-489f-858f-4b12139bdec2 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:39.187137 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10dddbe-cebd-489f-858f-4b12139bdec2 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:39.187137 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10dddbe-cebd-489f-858f-4b12139bdec2 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:39.188365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 15:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:06:39.193878 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce7fef-d815-45e0-af60-89db1f8a986c None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:39.194344 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce7fef-d815-45e0-af60-89db1f8a986c None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:39.194455 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce7fef-d815-45e0-af60-89db1f8a986c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:39.245509 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:39.266375 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-10ce7fef-d815-45e0-af60-89db1f8a986c None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:06:39.267571 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-10ce7fef-d815-45e0-af60-89db1f8a986c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:39.369305 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 15:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:06:39.650646 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10dddbe-cebd-489f-858f-4b12139bdec2 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:06:39.652219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-e10dddbe-cebd-489f-858f-4b12139bdec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:39.764611 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10dddbe-cebd-489f-858f-4b12139bdec2 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:06:39.806482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:39.843308 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22bb2d-fb94-435a-abb0-1821b25bc703 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:06:39.843940 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22bb2d-fb94-435a-abb0-1821b25bc703 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:39.844214 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22bb2d-fb94-435a-abb0-1821b25bc703 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:39.844488 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22bb2d-fb94-435a-abb0-1821b25bc703 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:39.846223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-5b22bb2d-fb94-435a-abb0-1821b25bc703 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:39.965024 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b22bb2d-fb94-435a-abb0-1821b25bc703 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:06:39.966404 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b22bb2d-fb94-435a-abb0-1821b25bc703 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:06:39.985153 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 722/1449] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:06:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:06:42.355161 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad75633-e275-40c9-ac82-41ec75bc4f17 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:42.355719 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad75633-e275-40c9-ac82-41ec75bc4f17 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:42.355719 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad75633-e275-40c9-ac82-41ec75bc4f17 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:42.357250 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 15:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:06:42.363854 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe66d8-5041-4eef-9091-aecd159746e6 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:42.364454 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe66d8-5041-4eef-9091-aecd159746e6 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:42.364454 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe66d8-5041-4eef-9091-aecd159746e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:42.439098 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-78fe66d8-5041-4eef-9091-aecd159746e6 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:06:42.440344 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-78fe66d8-5041-4eef-9091-aecd159746e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:42.464171 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:42.544822 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 15:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:06:42.825926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad75633-e275-40c9-ac82-41ec75bc4f17 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:06:42.827650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-2ad75633-e275-40c9-ac82-41ec75bc4f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:42.937170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad75633-e275-40c9-ac82-41ec75bc4f17 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:06:42.959131 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:43.018573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce635478-04ce-4778-96a1-e895e0b1a639 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:06:43.019202 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce635478-04ce-4778-96a1-e895e0b1a639 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:43.019475 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce635478-04ce-4778-96a1-e895e0b1a639 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:43.019738 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce635478-04ce-4778-96a1-e895e0b1a639 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:43.021539 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-ce635478-04ce-4778-96a1-e895e0b1a639 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:43.150450 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce635478-04ce-4778-96a1-e895e0b1a639 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:06:43.151930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce635478-04ce-4778-96a1-e895e0b1a639 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:06:43.172334 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 729/1452] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:06:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:06:45.595099 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe638e2e-525f-4fb4-8b95-1959cf8362cd None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:45.595758 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe638e2e-525f-4fb4-8b95-1959cf8362cd None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:45.595758 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe638e2e-525f-4fb4-8b95-1959cf8362cd None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:45.596905 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 15:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:06:45.603063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143b8e7-2351-4127-bdb1-a6c0628bdca1 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:45.603590 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143b8e7-2351-4127-bdb1-a6c0628bdca1 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:45.603707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143b8e7-2351-4127-bdb1-a6c0628bdca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:45.679080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:45.682281 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-c143b8e7-2351-4127-bdb1-a6c0628bdca1 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:06:45.684547 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-c143b8e7-2351-4127-bdb1-a6c0628bdca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:45.785421 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 15:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:06:46.070818 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe638e2e-525f-4fb4-8b95-1959cf8362cd None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:06:46.072407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-fe638e2e-525f-4fb4-8b95-1959cf8362cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:46.185208 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe638e2e-525f-4fb4-8b95-1959cf8362cd None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:06:46.259070 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:46.291019 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eadcee-1618-45e8-84e0-cf63c8bad9e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:06:46.291646 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eadcee-1618-45e8-84e0-cf63c8bad9e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:46.291985 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eadcee-1618-45e8-84e0-cf63c8bad9e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:46.292270 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eadcee-1618-45e8-84e0-cf63c8bad9e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:46.294015 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-58eadcee-1618-45e8-84e0-cf63c8bad9e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:46.429340 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eadcee-1618-45e8-84e0-cf63c8bad9e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:06:46.430725 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eadcee-1618-45e8-84e0-cf63c8bad9e7 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:06:46.450749 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 725/1455] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:06:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:06:48.758841 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58393d6e-67e2-486f-89fa-7ff505d1e93d None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:48.759491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58393d6e-67e2-486f-89fa-7ff505d1e93d None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:48.759491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58393d6e-67e2-486f-89fa-7ff505d1e93d None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:48.760614 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 15:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:06:48.766883 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f487c83-d349-428e-9a62-d92714d59b59 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:48.767421 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f487c83-d349-428e-9a62-d92714d59b59 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:48.767421 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f487c83-d349-428e-9a62-d92714d59b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:48.856236 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-1f487c83-d349-428e-9a62-d92714d59b59 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:06:48.859048 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-1f487c83-d349-428e-9a62-d92714d59b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:48.896702 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:48.975522 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 15:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:06:49.263738 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58393d6e-67e2-486f-89fa-7ff505d1e93d None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:06:49.265441 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-58393d6e-67e2-486f-89fa-7ff505d1e93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:49.388318 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58393d6e-67e2-486f-89fa-7ff505d1e93d None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:06:49.420517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:49.470323 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676ce60-c5ff-4a37-b2d1-52a6384e00d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:06:49.470974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676ce60-c5ff-4a37-b2d1-52a6384e00d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:49.471243 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676ce60-c5ff-4a37-b2d1-52a6384e00d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:49.471523 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676ce60-c5ff-4a37-b2d1-52a6384e00d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:49.473235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-f676ce60-c5ff-4a37-b2d1-52a6384e00d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:49.599275 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676ce60-c5ff-4a37-b2d1-52a6384e00d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:06:49.600964 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676ce60-c5ff-4a37-b2d1-52a6384e00d1 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:06:49.622672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 732/1458] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:06:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:06:52.024960 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56919e7-f2c0-45ec-bafa-85b027305124 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:52.025620 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56919e7-f2c0-45ec-bafa-85b027305124 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:52.025620 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56919e7-f2c0-45ec-bafa-85b027305124 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:52.026958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 15:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:06:52.032975 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9dbbb8-bf5a-456f-b474-baf87acd534f None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:52.033495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9dbbb8-bf5a-456f-b474-baf87acd534f None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:52.033598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9dbbb8-bf5a-456f-b474-baf87acd534f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:52.103417 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-2f9dbbb8-bf5a-456f-b474-baf87acd534f None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:06:52.104828 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-2f9dbbb8-bf5a-456f-b474-baf87acd534f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:52.114578 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:52.208692 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 15:06:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:06:52.488029 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56919e7-f2c0-45ec-bafa-85b027305124 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:06:52.490143 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-a56919e7-f2c0-45ec-bafa-85b027305124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:52.612006 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56919e7-f2c0-45ec-bafa-85b027305124 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:06:52.671329 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:52.690465 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc140944-c4e2-4984-a772-b2d91023b080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:06:52.691078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc140944-c4e2-4984-a772-b2d91023b080 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:52.691347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc140944-c4e2-4984-a772-b2d91023b080 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:52.691611 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc140944-c4e2-4984-a772-b2d91023b080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:52.693245 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-bc140944-c4e2-4984-a772-b2d91023b080 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:52.819702 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc140944-c4e2-4984-a772-b2d91023b080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:06:52.821267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc140944-c4e2-4984-a772-b2d91023b080 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:06:52.841839 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 728/1461] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:06:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:06:55.089911 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d14f5-8a0a-48ad-9434-68603e3ccd59 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:55.090443 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d14f5-8a0a-48ad-9434-68603e3ccd59 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:55.090443 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d14f5-8a0a-48ad-9434-68603e3ccd59 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:55.091898 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 15:06:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:06:55.098930 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e22c30-6c4a-43cf-8c6f-07efe8b755e6 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:55.099508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e22c30-6c4a-43cf-8c6f-07efe8b755e6 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:55.099508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e22c30-6c4a-43cf-8c6f-07efe8b755e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:55.294218 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-68e22c30-6c4a-43cf-8c6f-07efe8b755e6 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:06:55.295470 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-68e22c30-6c4a-43cf-8c6f-07efe8b755e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:55.341240 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:55.402486 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 15:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:06:55.683415 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d14f5-8a0a-48ad-9434-68603e3ccd59 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:06:55.685017 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-bf2d14f5-8a0a-48ad-9434-68603e3ccd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:55.802003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d14f5-8a0a-48ad-9434-68603e3ccd59 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:06:55.843367 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:55.886945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85050a0-41ca-49b8-b6f7-8845a25f0836 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:06:55.887573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85050a0-41ca-49b8-b6f7-8845a25f0836 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:55.887851 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85050a0-41ca-49b8-b6f7-8845a25f0836 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:55.888116 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85050a0-41ca-49b8-b6f7-8845a25f0836 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:55.889803 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-a85050a0-41ca-49b8-b6f7-8845a25f0836 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:56.015897 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85050a0-41ca-49b8-b6f7-8845a25f0836 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:06:56.017363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85050a0-41ca-49b8-b6f7-8845a25f0836 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:06:56.037922 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 735/1464] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:06:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:06:58.352775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65666ab9-2ba4-466e-a69a-667b94f90acb None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:58.353439 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65666ab9-2ba4-466e-a69a-667b94f90acb None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:58.353770 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65666ab9-2ba4-466e-a69a-667b94f90acb None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:58.356261 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 15:06:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:06:58.365812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d28ec-adde-4f5a-b074-318bcf60ea07 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:58.366700 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d28ec-adde-4f5a-b074-318bcf60ea07 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:58.366700 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d28ec-adde-4f5a-b074-318bcf60ea07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:58.462630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-095d28ec-adde-4f5a-b074-318bcf60ea07 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:06:58.463936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-095d28ec-adde-4f5a-b074-318bcf60ea07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:58.490588 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:58.603760 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 15:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:06:58.886111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65666ab9-2ba4-466e-a69a-667b94f90acb None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:06:58.887666 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-65666ab9-2ba4-466e-a69a-667b94f90acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:59.001531 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65666ab9-2ba4-466e-a69a-667b94f90acb None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:06:59.081151 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:06:59.089517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d0e6d-8320-4743-953e-7cf2876378e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:06:59.090030 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d0e6d-8320-4743-953e-7cf2876378e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:06:59.090274 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d0e6d-8320-4743-953e-7cf2876378e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:06:59.090543 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d0e6d-8320-4743-953e-7cf2876378e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:06:59.092169 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-aa3d0e6d-8320-4743-953e-7cf2876378e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:06:59.225687 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3d0e6d-8320-4743-953e-7cf2876378e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:06:59.227048 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3d0e6d-8320-4743-953e-7cf2876378e7 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:06:59.248560 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 731/1467] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:06:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:07:01.486075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce51463-3365-4709-88e2-89b69eca182a None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:07:01.486953 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce51463-3365-4709-88e2-89b69eca182a None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:07:01.486953 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce51463-3365-4709-88e2-89b69eca182a None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:07:01.488539 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 15:07:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:07:01.497957 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfb629-1342-4b8e-bcad-19425ddae75f None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:07:01.499218 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfb629-1342-4b8e-bcad-19425ddae75f None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:07:01.499499 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfb629-1342-4b8e-bcad-19425ddae75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:07:01.569396 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-85cfb629-1342-4b8e-bcad-19425ddae75f None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:07:01.570607 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-85cfb629-1342-4b8e-bcad-19425ddae75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:07:01.620184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:07:01.686008 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 15:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:07:01.971095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce51463-3365-4709-88e2-89b69eca182a None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:07:01.972774 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-1ce51463-3365-4709-88e2-89b69eca182a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:07:02.082448 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce51463-3365-4709-88e2-89b69eca182a None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:07:02.142908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:07:02.181817 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ac846-81fa-4bbf-a301-7f327845a78c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:07:02.182645 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ac846-81fa-4bbf-a301-7f327845a78c service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:07:02.183060 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ac846-81fa-4bbf-a301-7f327845a78c service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:07:02.183471 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ac846-81fa-4bbf-a301-7f327845a78c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:07:02.186177 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-956ac846-81fa-4bbf-a301-7f327845a78c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:07:02.320524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956ac846-81fa-4bbf-a301-7f327845a78c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:07:02.322095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956ac846-81fa-4bbf-a301-7f327845a78c service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:07:02.346693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 738/1470] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:07:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:07:11.684176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=22 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:07:12.343469 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=66 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:07:22.567915 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd70e2b-bb6d-47ff-b509-f6e69c79f863 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:07:22.568510 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd70e2b-bb6d-47ff-b509-f6e69c79f863 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:07:22.568510 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd70e2b-bb6d-47ff-b509-f6e69c79f863 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:07:22.569736 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 733/1471] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:07:22.575466 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986147e-bb4b-46ac-b865-326b79bc4b30 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:07:22.576056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986147e-bb4b-46ac-b865-326b79bc4b30 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:07:22.576056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986147e-bb4b-46ac-b865-326b79bc4b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:07:22.653686 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-9986147e-bb4b-46ac-b865-326b79bc4b30 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:07:22.654922 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-9986147e-bb4b-46ac-b865-326b79bc4b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:07:22.764710 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 739/1472] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:07:23.094875 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0298d-e2f4-42da-aaff-6ee9e213b9a6 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:07:23.095186 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0298d-e2f4-42da-aaff-6ee9e213b9a6 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:07:23.095436 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0298d-e2f4-42da-aaff-6ee9e213b9a6 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:07:23.096841 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 734/1473] 10.222.0.44 () {54 vars in 966 bytes} [Sun Sep 11 15:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:07:23.102585 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f077a-8e37-439d-bfdb-e51bde661583 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:07:23.102885 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f077a-8e37-439d-bfdb-e51bde661583 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:07:23.103154 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f077a-8e37-439d-bfdb-e51bde661583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:07:23.195996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-6e4f077a-8e37-439d-bfdb-e51bde661583 None None] MFA Rules not processed for user `a1fe5293447346eaafd16818e5bd12f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:07:23.197239 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-6e4f077a-8e37-439d-bfdb-e51bde661583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:07:23.220026 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:07:23.281636 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 740/1474] 10.222.0.44 () {58 vars in 1063 bytes} [Sun Sep 11 15:07:23 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:07:23.289814 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0298d-e2f4-42da-aaff-6ee9e213b9a6 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:07:23.291354 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-42d0298d-e2f4-42da-aaff-6ee9e213b9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:07:23.405561 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0298d-e2f4-42da-aaff-6ee9e213b9a6 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:07:23.490182 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf5581-85c8-4de4-9263-219610007ff1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fe5293447346eaafd16818e5bd12f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:07:23.490789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf5581-85c8-4de4-9263-219610007ff1 service glance] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:07:23.491055 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf5581-85c8-4de4-9263-219610007ff1 service glance] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:07:23.491312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf5581-85c8-4de4-9263-219610007ff1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:07:23.492902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-12bf5581-85c8-4de4-9263-219610007ff1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:07:23.553062 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:07:23.619961 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bf5581-85c8-4de4-9263-219610007ff1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:07:23.621437 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bf5581-85c8-4de4-9263-219610007ff1 service glance] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:07:23.642374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 735/1475] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:07:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:07:25.959467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5376fa-623a-4418-8e65-102813821436 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:07:25.960086 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5376fa-623a-4418-8e65-102813821436 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:07:25.960086 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5376fa-623a-4418-8e65-102813821436 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:07:25.961409 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 741/1476] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:07:25.967277 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba58f1-4fbc-45af-b02c-d9aeb32e71ca None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:07:25.967786 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba58f1-4fbc-45af-b02c-d9aeb32e71ca None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:07:25.967786 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba58f1-4fbc-45af-b02c-d9aeb32e71ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:07:26.038483 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-e8ba58f1-4fbc-45af-b02c-d9aeb32e71ca None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:07:26.039734 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-e8ba58f1-4fbc-45af-b02c-d9aeb32e71ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:07:26.141346 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 736/1477] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:07:26.408932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5376fa-623a-4418-8e65-102813821436 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:07:26.410506 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-fd5376fa-623a-4418-8e65-102813821436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:07:26.543229 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5376fa-623a-4418-8e65-102813821436 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:07:26.594852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:07:26.641245 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a984f175-ff04-4855-85d7-70d415fe9554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:07:26.641878 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a984f175-ff04-4855-85d7-70d415fe9554 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:07:26.642150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a984f175-ff04-4855-85d7-70d415fe9554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:07:26.642417 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a984f175-ff04-4855-85d7-70d415fe9554 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:07:26.643067 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a984f175-ff04-4855-85d7-70d415fe9554 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:07:26.644955 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: WARNING py.warnings [None req-a984f175-ff04-4855-85d7-70d415fe9554 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 15:07:26.644955 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: warnings.warn(msg) Sep 11 15:07:26.644955 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: Sep 11 15:07:26.646184 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a984f175-ff04-4855-85d7-70d415fe9554 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:07:26.668798 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 742/1478] 10.222.0.44 () {56 vars in 1108 bytes} [Sun Sep 11 15:07:26 2022] GET /identity/v3/projects => generated 2553 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:07:28.867469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9468aa-d245-4cec-9087-0d87190347ba None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:07:28.868020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9468aa-d245-4cec-9087-0d87190347ba None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:07:28.868020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9468aa-d245-4cec-9087-0d87190347ba None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:07:28.869417 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 737/1479] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:07:28.874874 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e55a1-df8c-4592-a793-9003c28090b8 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:07:28.875318 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e55a1-df8c-4592-a793-9003c28090b8 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:07:28.875421 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e55a1-df8c-4592-a793-9003c28090b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:07:28.942635 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-a63e55a1-df8c-4592-a793-9003c28090b8 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:07:28.943850 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-a63e55a1-df8c-4592-a793-9003c28090b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:07:29.046550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 743/1480] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:07:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:07:29.341789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9468aa-d245-4cec-9087-0d87190347ba None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:07:29.343296 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-7a9468aa-d245-4cec-9087-0d87190347ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:07:29.353299 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:07:29.502661 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9468aa-d245-4cec-9087-0d87190347ba None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:07:29.591198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:07:29.626393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25c926-e164-416a-b335-7a72ebd5ea0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:07:29.626997 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25c926-e164-416a-b335-7a72ebd5ea0d service nova] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:07:29.627276 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25c926-e164-416a-b335-7a72ebd5ea0d service nova] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:07:29.627621 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25c926-e164-416a-b335-7a72ebd5ea0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:07:29.629272 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-4f25c926-e164-416a-b335-7a72ebd5ea0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:07:29.724860 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25c926-e164-416a-b335-7a72ebd5ea0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:07:29.726489 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25c926-e164-416a-b335-7a72ebd5ea0d service nova] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:07:29.746040 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 738/1481] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:07:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:07:32.081300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df4f2d-13a9-42b6-b69d-03b0b3c8ae10 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:07:32.081883 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df4f2d-13a9-42b6-b69d-03b0b3c8ae10 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:07:32.081883 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df4f2d-13a9-42b6-b69d-03b0b3c8ae10 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:07:32.083172 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 744/1482] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:07:32.089240 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe6358-a9ca-4922-9b31-2dace982e02d None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:07:32.089733 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe6358-a9ca-4922-9b31-2dace982e02d None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:07:32.089733 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe6358-a9ca-4922-9b31-2dace982e02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:07:32.158733 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-aabe6358-a9ca-4922-9b31-2dace982e02d None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:07:32.159940 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-aabe6358-a9ca-4922-9b31-2dace982e02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:07:32.210926 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:07:32.271111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 739/1483] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:07:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:07:32.561100 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df4f2d-13a9-42b6-b69d-03b0b3c8ae10 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:07:32.562681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-a2df4f2d-13a9-42b6-b69d-03b0b3c8ae10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:07:32.588680 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:07:32.676254 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df4f2d-13a9-42b6-b69d-03b0b3c8ae10 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:07:32.758349 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a475cc-397c-4163-ae8c-aacdb4616fc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:07:32.758972 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a475cc-397c-4163-ae8c-aacdb4616fc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:07:32.759249 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a475cc-397c-4163-ae8c-aacdb4616fc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:07:32.759518 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a475cc-397c-4163-ae8c-aacdb4616fc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:07:32.761161 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-28a475cc-397c-4163-ae8c-aacdb4616fc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:07:32.968741 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:07:33.009498 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a475cc-397c-4163-ae8c-aacdb4616fc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:07:33.010945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a475cc-397c-4163-ae8c-aacdb4616fc8 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:07:33.029743 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 745/1484] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:07:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:07:35.238282 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c7ae3-c6a3-4c32-9bbb-b2b381ab80dd None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:07:35.239707 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c7ae3-c6a3-4c32-9bbb-b2b381ab80dd None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:07:35.239824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c7ae3-c6a3-4c32-9bbb-b2b381ab80dd None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:07:35.241565 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 740/1485] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:07:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 15:07:35.247587 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc237e6-3ce7-4b62-9e5f-92bac0f89cab None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:07:35.248309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc237e6-3ce7-4b62-9e5f-92bac0f89cab None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:07:35.248309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc237e6-3ce7-4b62-9e5f-92bac0f89cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:07:35.320243 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-4cc237e6-3ce7-4b62-9e5f-92bac0f89cab None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:07:35.321489 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-4cc237e6-3ce7-4b62-9e5f-92bac0f89cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:07:35.420933 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 746/1486] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:07:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:07:35.709316 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c7ae3-c6a3-4c32-9bbb-b2b381ab80dd None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:07:35.710860 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-8b9c7ae3-c6a3-4c32-9bbb-b2b381ab80dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:07:35.817827 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c7ae3-c6a3-4c32-9bbb-b2b381ab80dd None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:07:35.897025 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:07:35.898542 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a79682c-59fe-4e32-bac7-452e0eff7650 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:07:35.899147 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79682c-59fe-4e32-bac7-452e0eff7650 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:07:35.899411 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79682c-59fe-4e32-bac7-452e0eff7650 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:07:35.899688 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79682c-59fe-4e32-bac7-452e0eff7650 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:07:35.901853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-4a79682c-59fe-4e32-bac7-452e0eff7650 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:07:36.020428 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a79682c-59fe-4e32-bac7-452e0eff7650 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:07:36.021961 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a79682c-59fe-4e32-bac7-452e0eff7650 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:07:36.040943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 741/1487] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:07:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:07:38.289573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023972f-43ca-4e11-a0f4-98ac53434fb7 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:07:38.290133 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023972f-43ca-4e11-a0f4-98ac53434fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:07:38.290133 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023972f-43ca-4e11-a0f4-98ac53434fb7 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:07:38.291461 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 747/1488] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:07:38.297326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3cff7-730e-424f-b009-b38951cf01af None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:07:38.297813 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3cff7-730e-424f-b009-b38951cf01af None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:07:38.297813 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3cff7-730e-424f-b009-b38951cf01af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:07:38.372548 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-f3c3cff7-730e-424f-b009-b38951cf01af None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:07:38.373891 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f3c3cff7-730e-424f-b009-b38951cf01af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:07:38.433654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:07:38.479122 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 742/1489] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:07:38.768931 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023972f-43ca-4e11-a0f4-98ac53434fb7 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:07:38.770476 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-0023972f-43ca-4e11-a0f4-98ac53434fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:07:38.807681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:07:38.877981 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023972f-43ca-4e11-a0f4-98ac53434fb7 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:07:38.954598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db2f77-132c-4cce-8270-6e7742092592 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:07:38.955215 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db2f77-132c-4cce-8270-6e7742092592 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:07:38.955487 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db2f77-132c-4cce-8270-6e7742092592 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:07:38.955758 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db2f77-132c-4cce-8270-6e7742092592 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:07:38.957390 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-c6db2f77-132c-4cce-8270-6e7742092592 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:07:39.045019 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:07:39.074809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db2f77-132c-4cce-8270-6e7742092592 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:07:39.076383 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db2f77-132c-4cce-8270-6e7742092592 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:07:39.095082 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 748/1490] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:07:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:07:41.512506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4fcdd-10d9-4626-9a76-099c61eab42a None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:07:41.513262 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4fcdd-10d9-4626-9a76-099c61eab42a None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:07:41.513262 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4fcdd-10d9-4626-9a76-099c61eab42a None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:07:41.514466 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 743/1491] 10.222.0.44 () {54 vars in 927 bytes} [Sun Sep 11 15:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:07:41.520685 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c7c2c-31fa-4ac0-9dc3-eca41fb6f63f None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:07:41.521198 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c7c2c-31fa-4ac0-9dc3-eca41fb6f63f None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:07:41.521198 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c7c2c-31fa-4ac0-9dc3-eca41fb6f63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:07:41.597666 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-288c7c2c-31fa-4ac0-9dc3-eca41fb6f63f None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:07:41.598895 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-288c7c2c-31fa-4ac0-9dc3-eca41fb6f63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:07:41.701232 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 749/1492] 10.222.0.44 () {58 vars in 1024 bytes} [Sun Sep 11 15:07:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:07:41.988001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4fcdd-10d9-4626-9a76-099c61eab42a None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:07:41.989620 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b8d4fcdd-10d9-4626-9a76-099c61eab42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:07:42.100056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4fcdd-10d9-4626-9a76-099c61eab42a None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:07:42.176281 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468aa13b-6592-498a-9957-bf179f435b3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:07:42.176897 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468aa13b-6592-498a-9957-bf179f435b3b service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:07:42.177171 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468aa13b-6592-498a-9957-bf179f435b3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:07:42.177455 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468aa13b-6592-498a-9957-bf179f435b3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:07:42.179066 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-468aa13b-6592-498a-9957-bf179f435b3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:07:42.188270 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:07:42.297010 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468aa13b-6592-498a-9957-bf179f435b3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:07:42.298481 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468aa13b-6592-498a-9957-bf179f435b3b service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:07:42.316882 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 744/1493] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:07:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:07:51.699504 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=82 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:07:52.314618 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=50 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:08:55.306222 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b432d-f420-40be-9c3f-87215f366f6b None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:08:55.306820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b432d-f420-40be-9c3f-87215f366f6b None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:08:55.306820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b432d-f420-40be-9c3f-87215f366f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:08:55.389445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-796b432d-f420-40be-9c3f-87215f366f6b None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:08:55.390660 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-796b432d-f420-40be-9c3f-87215f366f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:08:55.501221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 750/1494] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:08:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:08:55.509671 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9eb7b1-13d1-46cb-8592-f7a3f65c3370 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:08:55.510323 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9eb7b1-13d1-46cb-8592-f7a3f65c3370 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:08:55.510323 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9eb7b1-13d1-46cb-8592-f7a3f65c3370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:08:55.593960 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-ed9eb7b1-13d1-46cb-8592-f7a3f65c3370 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:08:55.595123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-ed9eb7b1-13d1-46cb-8592-f7a3f65c3370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:08:55.696721 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 745/1495] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:08:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:08:55.701845 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b432d-f420-40be-9c3f-87215f366f6b None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:08:55.703112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-796b432d-f420-40be-9c3f-87215f366f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:08:55.715428 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b432d-f420-40be-9c3f-87215f366f6b None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:08:55.781173 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:08:55.816942 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2b9dc-984f-4ccf-8a4e-63b5a01b70ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:08:55.817582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2b9dc-984f-4ccf-8a4e-63b5a01b70ef admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:08:55.817856 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2b9dc-984f-4ccf-8a4e-63b5a01b70ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:08:55.818119 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2b9dc-984f-4ccf-8a4e-63b5a01b70ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:08:55.818766 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c2b9dc-984f-4ccf-8a4e-63b5a01b70ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:08:55.819058 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c2b9dc-984f-4ccf-8a4e-63b5a01b70ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:08:55.820072 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: WARNING py.warnings [None req-a7c2b9dc-984f-4ccf-8a4e-63b5a01b70ef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 11 15:08:55.820072 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: warnings.warn(msg) Sep 11 15:08:55.820072 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: Sep 11 15:08:55.820956 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c2b9dc-984f-4ccf-8a4e-63b5a01b70ef admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:08:55.832922 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 751/1496] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:08:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:08:55.841687 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9eb7b1-13d1-46cb-8592-f7a3f65c3370 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:08:55.851942 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9eb7b1-13d1-46cb-8592-f7a3f65c3370 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:08:55.918897 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:08:55.954666 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb48892-5a19-4600-928e-0f0bc50b644d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:08:55.955295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb48892-5a19-4600-928e-0f0bc50b644d admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:08:55.955571 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb48892-5a19-4600-928e-0f0bc50b644d admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:08:55.955838 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb48892-5a19-4600-928e-0f0bc50b644d admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:08:55.956811 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb48892-5a19-4600-928e-0f0bc50b644d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:08:55.958680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: WARNING py.warnings [None req-acb48892-5a19-4600-928e-0f0bc50b644d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 15:08:55.958680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: warnings.warn(msg) Sep 11 15:08:55.958680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: Sep 11 15:08:55.959855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb48892-5a19-4600-928e-0f0bc50b644d admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:08:56.018520 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 746/1497] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:08:55 2022] POST /identity/v3/projects => generated 379 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:08:56.023083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2b9dc-984f-4ccf-8a4e-63b5a01b70ef admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:08:56.024627 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-a7c2b9dc-984f-4ccf-8a4e-63b5a01b70ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:08:56.059399 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2b9dc-984f-4ccf-8a4e-63b5a01b70ef admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:08:56.161377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2d3c6-01ab-4895-b17b-5ea9ffd92da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:08:56.161893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2d3c6-01ab-4895-b17b-5ea9ffd92da4 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:08:56.162167 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2d3c6-01ab-4895-b17b-5ea9ffd92da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:08:56.162420 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2d3c6-01ab-4895-b17b-5ea9ffd92da4 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:08:56.163046 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2d3c6-01ab-4895-b17b-5ea9ffd92da4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:08:56.164063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: WARNING py.warnings [None req-eec2d3c6-01ab-4895-b17b-5ea9ffd92da4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 15:08:56.164063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: warnings.warn(msg) Sep 11 15:08:56.164063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: Sep 11 15:08:56.165285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2d3c6-01ab-4895-b17b-5ea9ffd92da4 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:08:56.212567 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 752/1498] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:08:56 2022] POST /identity/v3/users => generated 290 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:08:56.217610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb48892-5a19-4600-928e-0f0bc50b644d admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:08:56.219147 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-acb48892-5a19-4600-928e-0f0bc50b644d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:08:56.284281 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb48892-5a19-4600-928e-0f0bc50b644d admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:08:56.371938 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:08:56.383240 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd50588-2b4b-434e-b978-48ae5944f8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:08:56.383855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd50588-2b4b-434e-b978-48ae5944f8a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:08:56.384138 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd50588-2b4b-434e-b978-48ae5944f8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:08:56.384409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd50588-2b4b-434e-b978-48ae5944f8a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:08:56.385056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd50588-2b4b-434e-b978-48ae5944f8a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:08:56.386758 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: WARNING py.warnings [None req-dcd50588-2b4b-434e-b978-48ae5944f8a5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 11 15:08:56.386758 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: warnings.warn(msg) Sep 11 15:08:56.386758 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: Sep 11 15:08:56.387676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd50588-2b4b-434e-b978-48ae5944f8a5 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:08:56.400726 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 747/1499] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:08:56.405016 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2d3c6-01ab-4895-b17b-5ea9ffd92da4 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:08:56.406586 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-eec2d3c6-01ab-4895-b17b-5ea9ffd92da4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:08:56.418342 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2d3c6-01ab-4895-b17b-5ea9ffd92da4 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:08:56.445232 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:08:56.522201 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10d0b1-4cc2-412a-86c9-5a071690d6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:08:56.522818 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10d0b1-4cc2-412a-86c9-5a071690d6e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:08:56.523092 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10d0b1-4cc2-412a-86c9-5a071690d6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:08:56.523359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10d0b1-4cc2-412a-86c9-5a071690d6e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:08:56.524012 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10d0b1-4cc2-412a-86c9-5a071690d6e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:08:56.525030 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: WARNING py.warnings [None req-6c10d0b1-4cc2-412a-86c9-5a071690d6e8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 11 15:08:56.525030 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: warnings.warn(msg) Sep 11 15:08:56.525030 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: Sep 11 15:08:56.525957 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10d0b1-4cc2-412a-86c9-5a071690d6e8 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:08:56.538824 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 753/1500] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:08:56.543271 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd50588-2b4b-434e-b978-48ae5944f8a5 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:08:56.544815 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-dcd50588-2b4b-434e-b978-48ae5944f8a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:08:56.579756 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd50588-2b4b-434e-b978-48ae5944f8a5 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:08:56.680769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9241df-b4f3-4644-935a-0a240846ed38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:08:56.681375 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9241df-b4f3-4644-935a-0a240846ed38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:08:56.681652 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9241df-b4f3-4644-935a-0a240846ed38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:08:56.681920 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9241df-b4f3-4644-935a-0a240846ed38 admin admin] PATH_INFO: `/v3/projects/99ad4c1281cb4cca8fa342781c138fed/users/6b2e7dab3cdb4fc4bc73285e62c6e9cb/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:08:56.703801 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9241df-b4f3-4644-935a-0a240846ed38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99ad4c1281cb4cca8fa342781c138fed, user_id=6b2e7dab3cdb4fc4bc73285e62c6e9cb, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:08:56.704810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: WARNING py.warnings [None req-bd9241df-b4f3-4644-935a-0a240846ed38 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 15:08:56.704810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: warnings.warn(msg) Sep 11 15:08:56.704810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: Sep 11 15:08:56.707135 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9241df-b4f3-4644-935a-0a240846ed38 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:08:56.736411 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 748/1501] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:08:56 2022] PUT /identity/v3/projects/99ad4c1281cb4cca8fa342781c138fed/users/6b2e7dab3cdb4fc4bc73285e62c6e9cb/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:08:56.749585 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2ed98-164f-4b05-8243-8570aaeab8f0 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:08:56.749849 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2ed98-164f-4b05-8243-8570aaeab8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:08:56.750126 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2ed98-164f-4b05-8243-8570aaeab8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:08:56.808858 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-39c2ed98-164f-4b05-8243-8570aaeab8f0 None None] MFA Rules not processed for user `6b2e7dab3cdb4fc4bc73285e62c6e9cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:08:56.810106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-39c2ed98-164f-4b05-8243-8570aaeab8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:08:56.894560 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 754/1502] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:08:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:08:56.910734 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920db04b-bd7b-418e-90c5-35bbe0a978cf None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:08:56.911017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920db04b-bd7b-418e-90c5-35bbe0a978cf None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:08:56.911291 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920db04b-bd7b-418e-90c5-35bbe0a978cf None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:08:56.912675 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 749/1503] 10.222.0.44 () {54 vars in 964 bytes} [Sun Sep 11 15:08:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:08:56.918259 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafbdca-9d54-4490-8001-b24a7e6adc35 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:08:56.918545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafbdca-9d54-4490-8001-b24a7e6adc35 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:08:56.918813 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafbdca-9d54-4490-8001-b24a7e6adc35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:08:56.960529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:08:57.016859 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-fdafbdca-9d54-4490-8001-b24a7e6adc35 None None] MFA Rules not processed for user `8b97e2542db54900a99f6f44b78a5e24`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:08:57.018123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-fdafbdca-9d54-4490-8001-b24a7e6adc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:08:57.098684 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 755/1504] 10.222.0.44 () {58 vars in 1061 bytes} [Sun Sep 11 15:08:56 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:08:57.109569 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920db04b-bd7b-418e-90c5-35bbe0a978cf None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:08:57.111977 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-920db04b-bd7b-418e-90c5-35bbe0a978cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:08:57.169708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:08:57.225759 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920db04b-bd7b-418e-90c5-35bbe0a978cf None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:08:57.303437 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562ca83-42f1-4452-a9e1-55c425a83d83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:08:57.304044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562ca83-42f1-4452-a9e1-55c425a83d83 service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:08:57.304319 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562ca83-42f1-4452-a9e1-55c425a83d83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:08:57.304584 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562ca83-42f1-4452-a9e1-55c425a83d83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:08:57.306183 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-4562ca83-42f1-4452-a9e1-55c425a83d83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:08:57.425355 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4562ca83-42f1-4452-a9e1-55c425a83d83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:08:57.426802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4562ca83-42f1-4452-a9e1-55c425a83d83 service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:08:57.430359 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:08:57.445732 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 750/1505] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:08:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:08:57.492399 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6b56a-943c-43a5-b08f-34cf04582d32 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:08:57.492860 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6b56a-943c-43a5-b08f-34cf04582d32 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:08:57.492979 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6b56a-943c-43a5-b08f-34cf04582d32 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:08:57.494231 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 756/1506] 10.222.0.44 () {52 vars in 830 bytes} [Sun Sep 11 15:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:08:57.498314 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562ca83-42f1-4452-a9e1-55c425a83d83 service cinder] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:08:57.499527 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-4562ca83-42f1-4452-a9e1-55c425a83d83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:08:57.605273 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562ca83-42f1-4452-a9e1-55c425a83d83 service cinder] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:08:57.700255 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ba04c-b325-4e3c-82f5-a781decd63be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:08:57.700895 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ba04c-b325-4e3c-82f5-a781decd63be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:08:57.701178 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ba04c-b325-4e3c-82f5-a781decd63be admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:08:57.701474 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ba04c-b325-4e3c-82f5-a781decd63be admin admin] PATH_INFO: `/v3/users/6b2e7dab3cdb4fc4bc73285e62c6e9cb` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:08:57.715928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ba04c-b325-4e3c-82f5-a781decd63be admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b2e7dab3cdb4fc4bc73285e62c6e9cb)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:08:57.717046 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: WARNING py.warnings [None req-572ba04c-b325-4e3c-82f5-a781decd63be admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 15:08:57.717046 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: warnings.warn(msg) Sep 11 15:08:57.717046 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: Sep 11 15:08:57.718297 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ba04c-b325-4e3c-82f5-a781decd63be admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:08:57.728926 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:08:57.788109 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-572ba04c-b325-4e3c-82f5-a781decd63be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b2e7dab3cdb4fc4bc73285e62c6e9cb'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:08:57.799108 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-572ba04c-b325-4e3c-82f5-a781decd63be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b2e7dab3cdb4fc4bc73285e62c6e9cb'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:08:57.824635 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-572ba04c-b325-4e3c-82f5-a781decd63be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b2e7dab3cdb4fc4bc73285e62c6e9cb'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:08:57.825088 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-572ba04c-b325-4e3c-82f5-a781decd63be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b2e7dab3cdb4fc4bc73285e62c6e9cb'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:08:57.840181 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-572ba04c-b325-4e3c-82f5-a781decd63be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b2e7dab3cdb4fc4bc73285e62c6e9cb'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:08:57.846225 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 751/1507] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:08:57 2022] DELETE /identity/v3/users/6b2e7dab3cdb4fc4bc73285e62c6e9cb => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:08:57.850805 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83558117-101a-46ef-a9b2-3eeabe1c2b49 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:08:57.851083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83558117-101a-46ef-a9b2-3eeabe1c2b49 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:08:57.851358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83558117-101a-46ef-a9b2-3eeabe1c2b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:08:57.911817 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-83558117-101a-46ef-a9b2-3eeabe1c2b49 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:08:57.913248 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-83558117-101a-46ef-a9b2-3eeabe1c2b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:08:57.996663 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:08:58.010694 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 757/1508] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:08:58.021538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ba04c-b325-4e3c-82f5-a781decd63be admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:08:58.022874 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-572ba04c-b325-4e3c-82f5-a781decd63be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:08:58.127504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ba04c-b325-4e3c-82f5-a781decd63be admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:08:58.195617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:08:58.219088 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce26168-ebc2-42d3-95a2-ef248d6a56aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:08:58.219702 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce26168-ebc2-42d3-95a2-ef248d6a56aa service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:08:58.219971 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce26168-ebc2-42d3-95a2-ef248d6a56aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:08:58.220233 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce26168-ebc2-42d3-95a2-ef248d6a56aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:08:58.221912 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-4ce26168-ebc2-42d3-95a2-ef248d6a56aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:08:58.336680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce26168-ebc2-42d3-95a2-ef248d6a56aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:08:58.338157 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce26168-ebc2-42d3-95a2-ef248d6a56aa service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:08:58.357516 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 752/1509] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:08:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:08:58.697035 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83558117-101a-46ef-a9b2-3eeabe1c2b49 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:08:58.698301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-83558117-101a-46ef-a9b2-3eeabe1c2b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:08:58.710323 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83558117-101a-46ef-a9b2-3eeabe1c2b49 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:08:58.811196 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c16f1-7953-460e-8773-112d099f2d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:08:58.811852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c16f1-7953-460e-8773-112d099f2d0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:08:58.812124 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c16f1-7953-460e-8773-112d099f2d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:08:58.812395 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c16f1-7953-460e-8773-112d099f2d0d admin admin] PATH_INFO: `/v3/projects/99ad4c1281cb4cca8fa342781c138fed` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:08:58.823000 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c16f1-7953-460e-8773-112d099f2d0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=99ad4c1281cb4cca8fa342781c138fed)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:08:58.824042 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: WARNING py.warnings [None req-f30c16f1-7953-460e-8773-112d099f2d0d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 15:08:58.824042 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: warnings.warn(msg) Sep 11 15:08:58.824042 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: Sep 11 15:08:58.825278 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c16f1-7953-460e-8773-112d099f2d0d admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:08:58.888739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-f30c16f1-7953-460e-8773-112d099f2d0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99ad4c1281cb4cca8fa342781c138fed', 'request_id': 'req-f30c16f1-7953-460e-8773-112d099f2d0d'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:08:58.895826 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-f30c16f1-7953-460e-8773-112d099f2d0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99ad4c1281cb4cca8fa342781c138fed', 'request_id': 'req-f30c16f1-7953-460e-8773-112d099f2d0d'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:08:58.909500 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-f30c16f1-7953-460e-8773-112d099f2d0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99ad4c1281cb4cca8fa342781c138fed', 'request_id': 'req-f30c16f1-7953-460e-8773-112d099f2d0d'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:08:58.936917 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-f30c16f1-7953-460e-8773-112d099f2d0d admin admin] The token cache is being invalidate because project 99ad4c1281cb4cca8fa342781c138fed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:08:58.937234 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-f30c16f1-7953-460e-8773-112d099f2d0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:08:58.937666 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-f30c16f1-7953-460e-8773-112d099f2d0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:08:58.939102 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 758/1510] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:08:58 2022] DELETE /identity/v3/projects/99ad4c1281cb4cca8fa342781c138fed => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:09:08.355170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=62 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:09:08.934852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:10:25.673260 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0aef6-e8fa-4a15-a25a-a0876cdfa2cf None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:25.673856 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0aef6-e8fa-4a15-a25a-a0876cdfa2cf None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:25.673856 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0aef6-e8fa-4a15-a25a-a0876cdfa2cf None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:25.675113 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 15:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:10:25.683596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f3f9a-3822-4bed-b1ee-a3b802308e9e None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:25.684344 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f3f9a-3822-4bed-b1ee-a3b802308e9e None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:25.684494 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f3f9a-3822-4bed-b1ee-a3b802308e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:25.776720 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-7f2f3f9a-3822-4bed-b1ee-a3b802308e9e None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:10:25.778417 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-7f2f3f9a-3822-4bed-b1ee-a3b802308e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:10:25.896162 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 15:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:10:26.052317 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0aef6-e8fa-4a15-a25a-a0876cdfa2cf None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:10:26.056090 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-5fd0aef6-e8fa-4a15-a25a-a0876cdfa2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:10:26.192618 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0aef6-e8fa-4a15-a25a-a0876cdfa2cf None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:10:26.198735 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45529c-d0ff-434a-b215-49841a501901 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:10:26.199380 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45529c-d0ff-434a-b215-49841a501901 service nova] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:26.199661 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45529c-d0ff-434a-b215-49841a501901 service nova] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:26.199930 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45529c-d0ff-434a-b215-49841a501901 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:26.209174 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d45529c-d0ff-434a-b215-49841a501901 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:10:26.210783 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d45529c-d0ff-434a-b215-49841a501901 service nova] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:10:26.220733 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 754/1513] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:10:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:10:27.328403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02733cf0-ddd0-44f7-bd4f-75bcb62c05b3 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:27.329448 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02733cf0-ddd0-44f7-bd4f-75bcb62c05b3 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:27.329448 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02733cf0-ddd0-44f7-bd4f-75bcb62c05b3 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:27.330650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 15:10:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:10:27.337325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a9576-2317-41bf-9686-317050649240 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:27.337897 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a9576-2317-41bf-9686-317050649240 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:27.337897 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a9576-2317-41bf-9686-317050649240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:27.357651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-213a9576-2317-41bf-9686-317050649240 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:10:27.358858 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-213a9576-2317-41bf-9686-317050649240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:10:27.368103 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 15:10:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:10:27.538325 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02733cf0-ddd0-44f7-bd4f-75bcb62c05b3 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:10:27.557162 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02733cf0-ddd0-44f7-bd4f-75bcb62c05b3 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:10:27.563224 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb07136-e922-47d9-a15b-64add082c365 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:10:27.563827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb07136-e922-47d9-a15b-64add082c365 service nova] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:27.564107 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb07136-e922-47d9-a15b-64add082c365 service nova] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:27.564380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb07136-e922-47d9-a15b-64add082c365 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:27.572582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb07136-e922-47d9-a15b-64add082c365 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:10:27.574205 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb07136-e922-47d9-a15b-64add082c365 service nova] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:10:27.584837 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 761/1516] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:10:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:10:27.678222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffb53f-3bd9-4196-a251-d652b0df9cc3 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:27.678547 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffb53f-3bd9-4196-a251-d652b0df9cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:27.679003 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffb53f-3bd9-4196-a251-d652b0df9cc3 None None] PATH_INFO: `/v3` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:27.680381 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 756/1517] 10.222.0.46 () {54 vars in 925 bytes} [Sun Sep 11 15:10:27 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:10:27.687618 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b085c-3c18-4a16-b7b8-f575235a1643 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:27.687906 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b085c-3c18-4a16-b7b8-f575235a1643 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:27.688148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b085c-3c18-4a16-b7b8-f575235a1643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:27.753781 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-6f5b085c-3c18-4a16-b7b8-f575235a1643 None None] MFA Rules not processed for user `ead06362f8a2444bbfe1405cd602f1c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:10:27.754992 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-6f5b085c-3c18-4a16-b7b8-f575235a1643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:10:27.772558 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:10:27.828047 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 762/1518] 10.222.0.46 () {58 vars in 1015 bytes} [Sun Sep 11 15:10:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:10:31.467189 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453c683-546b-4e7f-b229-f76323a2e5fc None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:31.468030 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453c683-546b-4e7f-b229-f76323a2e5fc None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:31.468205 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453c683-546b-4e7f-b229-f76323a2e5fc None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:31.469813 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 757/1519] 10.222.0.44 () {54 vars in 961 bytes} [Sun Sep 11 15:10:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:10:31.476165 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107edd2-817c-4c3e-80c7-3928efd60635 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:31.476639 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107edd2-817c-4c3e-80c7-3928efd60635 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:31.476743 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107edd2-817c-4c3e-80c7-3928efd60635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:31.500003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-8107edd2-817c-4c3e-80c7-3928efd60635 None None] MFA Rules not processed for user `ead06362f8a2444bbfe1405cd602f1c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:10:31.501236 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-8107edd2-817c-4c3e-80c7-3928efd60635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:10:31.512917 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 763/1520] 10.222.0.44 () {58 vars in 1058 bytes} [Sun Sep 11 15:10:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:10:31.525462 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453c683-546b-4e7f-b229-f76323a2e5fc None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:10:31.548456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453c683-546b-4e7f-b229-f76323a2e5fc None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:10:31.553811 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4b2e6-ccd3-4f54-b2bf-61fcfb381b7e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead06362f8a2444bbfe1405cd602f1c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:10:31.554414 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4b2e6-ccd3-4f54-b2bf-61fcfb381b7e service placement] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:31.554690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4b2e6-ccd3-4f54-b2bf-61fcfb381b7e service placement] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:31.554965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4b2e6-ccd3-4f54-b2bf-61fcfb381b7e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:31.563912 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f4b2e6-ccd3-4f54-b2bf-61fcfb381b7e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:10:31.565473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f4b2e6-ccd3-4f54-b2bf-61fcfb381b7e service placement] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:10:31.575542 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 758/1521] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:10:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:10:39.074026 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d3812-c2e8-4aa2-94cf-d2fa0673f63b None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:39.074604 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d3812-c2e8-4aa2-94cf-d2fa0673f63b None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:39.074604 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d3812-c2e8-4aa2-94cf-d2fa0673f63b None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:39.076612 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 15:10:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:10:39.085430 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d78ee3-28dc-4c5c-b36a-11d27e231c55 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:39.086100 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d78ee3-28dc-4c5c-b36a-11d27e231c55 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:39.086100 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d78ee3-28dc-4c5c-b36a-11d27e231c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:39.108641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-61d78ee3-28dc-4c5c-b36a-11d27e231c55 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:10:39.109899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-61d78ee3-28dc-4c5c-b36a-11d27e231c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:10:39.119222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 15:10:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:10:39.274295 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d3812-c2e8-4aa2-94cf-d2fa0673f63b None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:10:39.292930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d3812-c2e8-4aa2-94cf-d2fa0673f63b None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:10:39.299121 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639e6d3-1667-4a7f-ae07-05a7d58797d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:10:39.299722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639e6d3-1667-4a7f-ae07-05a7d58797d8 service nova] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:39.300003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639e6d3-1667-4a7f-ae07-05a7d58797d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:39.300273 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639e6d3-1667-4a7f-ae07-05a7d58797d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:39.308673 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f639e6d3-1667-4a7f-ae07-05a7d58797d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:10:39.310277 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f639e6d3-1667-4a7f-ae07-05a7d58797d8 service nova] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:10:39.319680 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 765/1524] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:10:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:10:40.374690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452f3ad-f080-442e-9f68-5b66b61d4cb8 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:40.375466 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452f3ad-f080-442e-9f68-5b66b61d4cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:40.375466 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452f3ad-f080-442e-9f68-5b66b61d4cb8 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:40.377573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 15:10:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:10:40.387520 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63521d-efba-435b-9104-714d8ebb5059 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:40.388073 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63521d-efba-435b-9104-714d8ebb5059 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:40.388073 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63521d-efba-435b-9104-714d8ebb5059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:40.411034 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-dd63521d-efba-435b-9104-714d8ebb5059 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:10:40.412275 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-dd63521d-efba-435b-9104-714d8ebb5059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:10:40.421821 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 15:10:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:10:40.580206 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452f3ad-f080-442e-9f68-5b66b61d4cb8 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:10:40.598687 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452f3ad-f080-442e-9f68-5b66b61d4cb8 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:10:40.604655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c289a951-0d28-4054-a87d-3d6e7ce35747 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:10:40.605283 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289a951-0d28-4054-a87d-3d6e7ce35747 service nova] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:40.605554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289a951-0d28-4054-a87d-3d6e7ce35747 service nova] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:40.605817 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289a951-0d28-4054-a87d-3d6e7ce35747 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:40.614572 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c289a951-0d28-4054-a87d-3d6e7ce35747 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:10:40.616139 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c289a951-0d28-4054-a87d-3d6e7ce35747 service nova] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:10:40.625643 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 761/1527] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:10:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:10:44.582729 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8f783-a131-47e1-a786-aab8d203ee36 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:44.583545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8f783-a131-47e1-a786-aab8d203ee36 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:44.583545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8f783-a131-47e1-a786-aab8d203ee36 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:44.585483 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 15:10:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:10:44.590838 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac4d885-d6c5-4eff-a967-e09419a126d2 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:44.591281 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac4d885-d6c5-4eff-a967-e09419a126d2 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:44.591389 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac4d885-d6c5-4eff-a967-e09419a126d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:44.611553 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-fac4d885-d6c5-4eff-a967-e09419a126d2 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:10:44.612746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-fac4d885-d6c5-4eff-a967-e09419a126d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:10:44.621944 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 15:10:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:10:44.912292 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8f783-a131-47e1-a786-aab8d203ee36 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:10:44.915516 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-fda8f783-a131-47e1-a786-aab8d203ee36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:10:45.009374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8f783-a131-47e1-a786-aab8d203ee36 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:10:45.014882 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f231211-1df5-4362-a1ca-79967575c0c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:10:45.015496 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f231211-1df5-4362-a1ca-79967575c0c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:45.015771 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f231211-1df5-4362-a1ca-79967575c0c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:45.016037 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f231211-1df5-4362-a1ca-79967575c0c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:45.025795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f231211-1df5-4362-a1ca-79967575c0c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:10:45.027218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f231211-1df5-4362-a1ca-79967575c0c6 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:10:45.037600 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 768/1530] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:10:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:10:47.170068 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea273b8-f037-48bc-b75b-6d2c44655114 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:47.170939 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea273b8-f037-48bc-b75b-6d2c44655114 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:47.170939 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea273b8-f037-48bc-b75b-6d2c44655114 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:47.172230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 15:10:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:10:47.177586 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444db62-ea0d-451f-914a-04a28539834b None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:47.178041 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444db62-ea0d-451f-914a-04a28539834b None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:47.178138 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444db62-ea0d-451f-914a-04a28539834b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:47.199309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-3444db62-ea0d-451f-914a-04a28539834b None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:10:47.200447 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-3444db62-ea0d-451f-914a-04a28539834b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:10:47.209996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 15:10:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:10:47.505566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea273b8-f037-48bc-b75b-6d2c44655114 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:10:47.508793 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-2ea273b8-f037-48bc-b75b-6d2c44655114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:10:47.531374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea273b8-f037-48bc-b75b-6d2c44655114 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:10:47.535924 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6fbbbf-8916-4d70-b440-48d08bea8f4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:10:47.536521 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fbbbf-8916-4d70-b440-48d08bea8f4f service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:47.536792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fbbbf-8916-4d70-b440-48d08bea8f4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:47.537063 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fbbbf-8916-4d70-b440-48d08bea8f4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:47.546595 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6fbbbf-8916-4d70-b440-48d08bea8f4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:10:47.548024 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6fbbbf-8916-4d70-b440-48d08bea8f4f service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:10:47.557743 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 764/1533] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:10:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:10:51.232503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3202627-952e-47d3-aaee-af7e78efa7db None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:51.233168 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3202627-952e-47d3-aaee-af7e78efa7db None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:51.233168 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3202627-952e-47d3-aaee-af7e78efa7db None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:51.234362 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 15:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:10:51.240369 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32eecb1-81a9-4768-a42f-edff84be8eea None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:51.240871 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32eecb1-81a9-4768-a42f-edff84be8eea None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:51.240871 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32eecb1-81a9-4768-a42f-edff84be8eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:51.251030 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=81 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:10:51.270125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-e32eecb1-81a9-4768-a42f-edff84be8eea None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:10:51.271442 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-e32eecb1-81a9-4768-a42f-edff84be8eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:10:51.281456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 15:10:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:10:51.583455 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3202627-952e-47d3-aaee-af7e78efa7db None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:10:51.585650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-b3202627-952e-47d3-aaee-af7e78efa7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:10:51.656435 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:10:51.676260 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3202627-952e-47d3-aaee-af7e78efa7db None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:10:51.680673 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac032743-90f7-430f-b180-bf60edce46b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:10:51.681345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac032743-90f7-430f-b180-bf60edce46b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:51.681625 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac032743-90f7-430f-b180-bf60edce46b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:51.681880 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac032743-90f7-430f-b180-bf60edce46b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:51.694406 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac032743-90f7-430f-b180-bf60edce46b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:10:51.695993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac032743-90f7-430f-b180-bf60edce46b6 service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:10:51.706529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 771/1536] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:10:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:10:53.842533 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd8ca5-91da-418a-8beb-2446d89248a5 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:53.843407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd8ca5-91da-418a-8beb-2446d89248a5 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:53.843407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd8ca5-91da-418a-8beb-2446d89248a5 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:53.845548 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 15:10:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:10:53.852027 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c84e7-8c7e-4b88-b2fd-f6ac005b54f2 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:53.852767 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c84e7-8c7e-4b88-b2fd-f6ac005b54f2 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:53.852936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c84e7-8c7e-4b88-b2fd-f6ac005b54f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:53.876684 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-a47c84e7-8c7e-4b88-b2fd-f6ac005b54f2 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:10:53.877925 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-a47c84e7-8c7e-4b88-b2fd-f6ac005b54f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:10:53.887270 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 15:10:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:10:54.191358 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd8ca5-91da-418a-8beb-2446d89248a5 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:10:54.193304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-91cd8ca5-91da-418a-8beb-2446d89248a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:10:54.208717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd8ca5-91da-418a-8beb-2446d89248a5 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:10:54.212904 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f23c84-0917-4174-9b2a-382566df2e5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:10:54.213549 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f23c84-0917-4174-9b2a-382566df2e5d service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:10:54.213806 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f23c84-0917-4174-9b2a-382566df2e5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:10:54.214074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f23c84-0917-4174-9b2a-382566df2e5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:10:54.223931 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f23c84-0917-4174-9b2a-382566df2e5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:10:54.225383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f23c84-0917-4174-9b2a-382566df2e5d service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:10:54.234788 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 767/1539] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:10:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:11:03.871151 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=11 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:11:04.230664 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=8 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:17.089113 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e53d67-bb54-4bce-b678-e3a0a1add764 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.090479 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e53d67-bb54-4bce-b678-e3a0a1add764 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.090962 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e53d67-bb54-4bce-b678-e3a0a1add764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.118313 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-34e53d67-bb54-4bce-b678-e3a0a1add764 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:17.119956 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-34e53d67-bb54-4bce-b678-e3a0a1add764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:17.135235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 773/1540] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:17.145650 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b7dc5-8c7d-42de-8931-4a13332c89e5 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.146702 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b7dc5-8c7d-42de-8931-4a13332c89e5 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.147208 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b7dc5-8c7d-42de-8931-4a13332c89e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.165274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c9969-7fd0-447e-b87c-91c3bc33ebfa None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.165634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c9969-7fd0-447e-b87c-91c3bc33ebfa None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.165808 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c9969-7fd0-447e-b87c-91c3bc33ebfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.184824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-9b7b7dc5-8c7d-42de-8931-4a13332c89e5 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:17.191063 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-9b7b7dc5-8c7d-42de-8931-4a13332c89e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:17.206566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 768/1541] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:17.207004 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-383c9969-7fd0-447e-b87c-91c3bc33ebfa None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:17.208264 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-383c9969-7fd0-447e-b87c-91c3bc33ebfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:17.220995 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b7dc5-8c7d-42de-8931-4a13332c89e5 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:17.227665 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 774/1542] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:12:17.237534 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bced2-d29e-4abd-bff0-92957ad3df72 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.237873 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bced2-d29e-4abd-bff0-92957ad3df72 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.238090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bced2-d29e-4abd-bff0-92957ad3df72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.241313 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b7dc5-8c7d-42de-8931-4a13332c89e5 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:17.248598 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc63494-66d5-4897-811e-7bde93d4cb09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:17.249148 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc63494-66d5-4897-811e-7bde93d4cb09 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.249434 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc63494-66d5-4897-811e-7bde93d4cb09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.249700 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc63494-66d5-4897-811e-7bde93d4cb09 admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.252196 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc63494-66d5-4897-811e-7bde93d4cb09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:12:17.252492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc63494-66d5-4897-811e-7bde93d4cb09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:17.255586 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: WARNING py.warnings [None req-6cc63494-66d5-4897-811e-7bde93d4cb09 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 11 15:12:17.255586 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: warnings.warn(msg) Sep 11 15:12:17.255586 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: Sep 11 15:12:17.256485 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc63494-66d5-4897-811e-7bde93d4cb09 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:17.282575 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 769/1543] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:12:17.284463 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-cd7bced2-d29e-4abd-bff0-92957ad3df72 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:17.285937 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-cd7bced2-d29e-4abd-bff0-92957ad3df72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:17.287324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787ba3e-9c8e-4b1c-8347-cf312b1b5bff None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.287591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787ba3e-9c8e-4b1c-8347-cf312b1b5bff None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.287853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787ba3e-9c8e-4b1c-8347-cf312b1b5bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.301182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 775/1544] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:17.304240 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59656d6c-4689-4f33-820c-78ba4546e7cd None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.305758 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59656d6c-4689-4f33-820c-78ba4546e7cd None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.306035 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59656d6c-4689-4f33-820c-78ba4546e7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.332334 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-9787ba3e-9c8e-4b1c-8347-cf312b1b5bff None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:17.333574 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-9787ba3e-9c8e-4b1c-8347-cf312b1b5bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:17.347976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-59656d6c-4689-4f33-820c-78ba4546e7cd None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:17.349187 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-59656d6c-4689-4f33-820c-78ba4546e7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:17.352626 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 770/1545] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:17.354135 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef88f8-6a72-4bdf-acd9-da8aaa529780 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.354425 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef88f8-6a72-4bdf-acd9-da8aaa529780 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.354700 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef88f8-6a72-4bdf-acd9-da8aaa529780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.359918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 776/1546] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:17.360980 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59656d6c-4689-4f33-820c-78ba4546e7cd None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:17.371137 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59656d6c-4689-4f33-820c-78ba4546e7cd None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:17.375874 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beedca5e-d017-4ad6-a266-8a5cdc45d172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:17.376502 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedca5e-d017-4ad6-a266-8a5cdc45d172 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.376775 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedca5e-d017-4ad6-a266-8a5cdc45d172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.376976 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-6bef88f8-6a72-4bdf-acd9-da8aaa529780 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:17.377112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedca5e-d017-4ad6-a266-8a5cdc45d172 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.377689 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beedca5e-d017-4ad6-a266-8a5cdc45d172 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:17.378278 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-6bef88f8-6a72-4bdf-acd9-da8aaa529780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:17.378697 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: WARNING py.warnings [None req-beedca5e-d017-4ad6-a266-8a5cdc45d172 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 15:12:17.378697 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: warnings.warn(msg) Sep 11 15:12:17.378697 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: Sep 11 15:12:17.380085 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beedca5e-d017-4ad6-a266-8a5cdc45d172 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:17.386902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 771/1547] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:17.387859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef88f8-6a72-4bdf-acd9-da8aaa529780 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:17.397509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef88f8-6a72-4bdf-acd9-da8aaa529780 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:17.402142 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82088a2f-bb37-4e16-9201-457b838d9f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:17.402740 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82088a2f-bb37-4e16-9201-457b838d9f20 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.403018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82088a2f-bb37-4e16-9201-457b838d9f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.403559 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82088a2f-bb37-4e16-9201-457b838d9f20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.403924 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82088a2f-bb37-4e16-9201-457b838d9f20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:12:17.404221 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82088a2f-bb37-4e16-9201-457b838d9f20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:17.405869 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82088a2f-bb37-4e16-9201-457b838d9f20 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:17.418974 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 772/1548] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:12:17.420339 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8e0f9-9754-4dc3-a2dd-f9b9157bf853 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.420757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8e0f9-9754-4dc3-a2dd-f9b9157bf853 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.421166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8e0f9-9754-4dc3-a2dd-f9b9157bf853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.425237 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 777/1549] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:12:17 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:17.425413 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702bf4d5-93dd-4f7f-a694-dc6ec23b3542 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.425672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702bf4d5-93dd-4f7f-a694-dc6ec23b3542 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.425937 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702bf4d5-93dd-4f7f-a694-dc6ec23b3542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.442619 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-33f8e0f9-9754-4dc3-a2dd-f9b9157bf853 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:17.443909 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-33f8e0f9-9754-4dc3-a2dd-f9b9157bf853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:17.445949 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-702bf4d5-93dd-4f7f-a694-dc6ec23b3542 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:17.447126 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-702bf4d5-93dd-4f7f-a694-dc6ec23b3542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:17.508582 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 773/1550] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:17.509810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10bac2a-a601-4cf8-b61b-01e9a7a9af6c None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.510084 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10bac2a-a601-4cf8-b61b-01e9a7a9af6c None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.510353 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10bac2a-a601-4cf8-b61b-01e9a7a9af6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.517995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 778/1551] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:17.519254 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ac373-5411-4705-9d7f-60272415faa4 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.519534 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ac373-5411-4705-9d7f-60272415faa4 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.519805 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ac373-5411-4705-9d7f-60272415faa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.533199 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-a10bac2a-a601-4cf8-b61b-01e9a7a9af6c None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:17.534554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-a10bac2a-a601-4cf8-b61b-01e9a7a9af6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:17.539623 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-fe3ac373-5411-4705-9d7f-60272415faa4 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:17.540786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-fe3ac373-5411-4705-9d7f-60272415faa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:17.542662 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 774/1552] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:17.543961 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2441c-97cd-4a39-bb6f-a61deab734b6 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.544327 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2441c-97cd-4a39-bb6f-a61deab734b6 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.544659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2441c-97cd-4a39-bb6f-a61deab734b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.549806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 779/1553] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:17.551390 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f257291-e005-43c5-adc3-7d37351d4a27 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.551666 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f257291-e005-43c5-adc3-7d37351d4a27 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.551932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f257291-e005-43c5-adc3-7d37351d4a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.569099 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-e9f2441c-97cd-4a39-bb6f-a61deab734b6 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:17.570331 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-e9f2441c-97cd-4a39-bb6f-a61deab734b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:17.572002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-6f257291-e005-43c5-adc3-7d37351d4a27 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:17.573233 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-6f257291-e005-43c5-adc3-7d37351d4a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:17.578889 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 775/1554] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:17.579738 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2441c-97cd-4a39-bb6f-a61deab734b6 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:17.581691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 780/1555] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:17.585416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f257291-e005-43c5-adc3-7d37351d4a27 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:17.589858 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2441c-97cd-4a39-bb6f-a61deab734b6 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:17.594701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333022c-32fe-4057-9f03-3beaa54390ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:17.594983 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f257291-e005-43c5-adc3-7d37351d4a27 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:17.595331 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333022c-32fe-4057-9f03-3beaa54390ee admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.595599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333022c-32fe-4057-9f03-3beaa54390ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.595867 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333022c-32fe-4057-9f03-3beaa54390ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.596491 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333022c-32fe-4057-9f03-3beaa54390ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:17.598420 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333022c-32fe-4057-9f03-3beaa54390ee admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:17.599910 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa7341-4b52-45a1-864d-68259258829f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:17.600524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa7341-4b52-45a1-864d-68259258829f admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.600842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa7341-4b52-45a1-864d-68259258829f admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.601108 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa7341-4b52-45a1-864d-68259258829f admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.601781 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa7341-4b52-45a1-864d-68259258829f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:17.603704 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa7341-4b52-45a1-864d-68259258829f admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:17.627463 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 776/1556] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:12:17 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:17.628231 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 781/1557] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:12:17 2022] POST /identity/v3/users => generated 290 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:17.628920 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e66c16-a587-4ec6-aa06-edd132ceaaec None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.629292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e66c16-a587-4ec6-aa06-edd132ceaaec None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.629618 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e66c16-a587-4ec6-aa06-edd132ceaaec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.630143 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ab0fb-e60c-4d3b-ae1f-7499dd4b1522 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.630653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ab0fb-e60c-4d3b-ae1f-7499dd4b1522 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.631071 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ab0fb-e60c-4d3b-ae1f-7499dd4b1522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.649958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-93e66c16-a587-4ec6-aa06-edd132ceaaec None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:17.651146 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-93e66c16-a587-4ec6-aa06-edd132ceaaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:17.651305 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-2f7ab0fb-e60c-4d3b-ae1f-7499dd4b1522 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:17.652287 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-2f7ab0fb-e60c-4d3b-ae1f-7499dd4b1522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:17.717408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 782/1558] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:17.718676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3e46f-b496-44a2-8b1c-bbfe3a15208a None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.718956 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3e46f-b496-44a2-8b1c-bbfe3a15208a None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.719225 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3e46f-b496-44a2-8b1c-bbfe3a15208a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.722429 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 777/1559] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:17.723392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e66c16-a587-4ec6-aa06-edd132ceaaec None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:17.733093 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e66c16-a587-4ec6-aa06-edd132ceaaec None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:17.737753 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e205e-6960-4aa3-87e6-f156d64f6b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:17.738382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e205e-6960-4aa3-87e6-f156d64f6b24 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.738655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e205e-6960-4aa3-87e6-f156d64f6b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.738924 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e205e-6960-4aa3-87e6-f156d64f6b24 admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.739579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e205e-6960-4aa3-87e6-f156d64f6b24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:12:17.739878 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e205e-6960-4aa3-87e6-f156d64f6b24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:17.741002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-dea3e46f-b496-44a2-8b1c-bbfe3a15208a None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:17.741514 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e205e-6960-4aa3-87e6-f156d64f6b24 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:17.742277 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-dea3e46f-b496-44a2-8b1c-bbfe3a15208a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:17.750628 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 783/1560] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:17.751428 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea3e46f-b496-44a2-8b1c-bbfe3a15208a None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:17.760794 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 778/1561] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:12:17.760794 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e205e-6960-4aa3-87e6-f156d64f6b24 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:17.761224 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea3e46f-b496-44a2-8b1c-bbfe3a15208a None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:17.765920 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb8ebe-21cf-48a1-97aa-f62b8f9619a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:17.766624 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb8ebe-21cf-48a1-97aa-f62b8f9619a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.767002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb8ebe-21cf-48a1-97aa-f62b8f9619a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.767272 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb8ebe-21cf-48a1-97aa-f62b8f9619a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.767926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb8ebe-21cf-48a1-97aa-f62b8f9619a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:12:17.768232 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb8ebe-21cf-48a1-97aa-f62b8f9619a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:17.769893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb8ebe-21cf-48a1-97aa-f62b8f9619a4 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:17.771092 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e205e-6960-4aa3-87e6-f156d64f6b24 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:17.771506 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:17.776674 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb64d66-9dfb-4e13-b914-17ea103c7739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:17.777307 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb64d66-9dfb-4e13-b914-17ea103c7739 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.777578 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb64d66-9dfb-4e13-b914-17ea103c7739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.777849 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb64d66-9dfb-4e13-b914-17ea103c7739 admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.778476 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb64d66-9dfb-4e13-b914-17ea103c7739 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:12:17.778776 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb64d66-9dfb-4e13-b914-17ea103c7739 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:17.780363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb64d66-9dfb-4e13-b914-17ea103c7739 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:17.782039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:17.784331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 784/1562] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:12:17.785695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb8ebe-21cf-48a1-97aa-f62b8f9619a4 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:17.794143 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 779/1563] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:12:17.795401 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb64d66-9dfb-4e13-b914-17ea103c7739 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:17.796156 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb8ebe-21cf-48a1-97aa-f62b8f9619a4 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:17.800812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aade94c-6007-4690-8bb2-dabadaa936b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:17.801676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aade94c-6007-4690-8bb2-dabadaa936b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.801953 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aade94c-6007-4690-8bb2-dabadaa936b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.802220 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aade94c-6007-4690-8bb2-dabadaa936b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.802837 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aade94c-6007-4690-8bb2-dabadaa936b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:17.804733 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aade94c-6007-4690-8bb2-dabadaa936b1 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:17.810846 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb64d66-9dfb-4e13-b914-17ea103c7739 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:17.815823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c7c48-93d3-4f73-8113-33536a172e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:17.816427 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c7c48-93d3-4f73-8113-33536a172e4a admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.816701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c7c48-93d3-4f73-8113-33536a172e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.817007 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c7c48-93d3-4f73-8113-33536a172e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.817657 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c7c48-93d3-4f73-8113-33536a172e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:17.819425 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278c7c48-93d3-4f73-8113-33536a172e4a admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:17.827809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 785/1564] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:12:17 2022] POST /identity/v3/users => generated 282 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:17.828649 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aade94c-6007-4690-8bb2-dabadaa936b1 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:17.831436 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 780/1565] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:17.832297 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c7c48-93d3-4f73-8113-33536a172e4a admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:17.839977 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aade94c-6007-4690-8bb2-dabadaa936b1 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:17.842512 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c7c48-93d3-4f73-8113-33536a172e4a admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:17.844695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb42561-c1d8-42e5-80dc-754e5d9e3903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:17.845472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb42561-c1d8-42e5-80dc-754e5d9e3903 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.845745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb42561-c1d8-42e5-80dc-754e5d9e3903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.846018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb42561-c1d8-42e5-80dc-754e5d9e3903 admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.846654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb42561-c1d8-42e5-80dc-754e5d9e3903 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:12:17.846951 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb42561-c1d8-42e5-80dc-754e5d9e3903 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:17.847097 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a79bb4-a9e1-40b9-ada4-68e404b3f3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:17.847741 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a79bb4-a9e1-40b9-ada4-68e404b3f3e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.848020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a79bb4-a9e1-40b9-ada4-68e404b3f3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.848287 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a79bb4-a9e1-40b9-ada4-68e404b3f3e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.848544 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb42561-c1d8-42e5-80dc-754e5d9e3903 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:17.848977 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a79bb4-a9e1-40b9-ada4-68e404b3f3e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:12:17.849288 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a79bb4-a9e1-40b9-ada4-68e404b3f3e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:17.851303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a79bb4-a9e1-40b9-ada4-68e404b3f3e4 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:17.861073 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 786/1566] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:12:17.862094 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb42561-c1d8-42e5-80dc-754e5d9e3903 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:17.870582 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 781/1567] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:12:17.871519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a79bb4-a9e1-40b9-ada4-68e404b3f3e4 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:17.872818 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb42561-c1d8-42e5-80dc-754e5d9e3903 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:17.878354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707c6c5-c3ed-4867-97cb-4fb61831dc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:17.878970 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707c6c5-c3ed-4867-97cb-4fb61831dc37 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.879257 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707c6c5-c3ed-4867-97cb-4fb61831dc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.879528 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707c6c5-c3ed-4867-97cb-4fb61831dc37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.880199 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b707c6c5-c3ed-4867-97cb-4fb61831dc37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:12:17.880520 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b707c6c5-c3ed-4867-97cb-4fb61831dc37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:17.882009 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a79bb4-a9e1-40b9-ada4-68e404b3f3e4 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:17.882141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b707c6c5-c3ed-4867-97cb-4fb61831dc37 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:17.886578 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67befe43-64d9-4a88-8dd3-2095411ee2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:17.887170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67befe43-64d9-4a88-8dd3-2095411ee2d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.887442 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67befe43-64d9-4a88-8dd3-2095411ee2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.887707 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67befe43-64d9-4a88-8dd3-2095411ee2d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.888329 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67befe43-64d9-4a88-8dd3-2095411ee2d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:17.890269 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67befe43-64d9-4a88-8dd3-2095411ee2d7 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:17.895606 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 787/1568] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:12:17.902666 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707c6c5-c3ed-4867-97cb-4fb61831dc37 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:17.913113 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707c6c5-c3ed-4867-97cb-4fb61831dc37 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:17.917749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb5b3e-7d5e-44db-9aaf-d6764300a34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:17.917998 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 782/1569] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:12:17 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:17.918695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb5b3e-7d5e-44db-9aaf-d6764300a34f admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:17.918939 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67befe43-64d9-4a88-8dd3-2095411ee2d7 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:17.919039 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb5b3e-7d5e-44db-9aaf-d6764300a34f admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:17.919258 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb5b3e-7d5e-44db-9aaf-d6764300a34f admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:17.919885 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb5b3e-7d5e-44db-9aaf-d6764300a34f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:17.921815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb5b3e-7d5e-44db-9aaf-d6764300a34f admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:17.950394 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 788/1570] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:12:17 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:17.951722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb5b3e-7d5e-44db-9aaf-d6764300a34f admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:17.985002 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67befe43-64d9-4a88-8dd3-2095411ee2d7 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.013800 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb5b3e-7d5e-44db-9aaf-d6764300a34f admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.018889 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24460798-c234-4f92-81f4-f22223090dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.019496 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24460798-c234-4f92-81f4-f22223090dab admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.019773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24460798-c234-4f92-81f4-f22223090dab admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.020042 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24460798-c234-4f92-81f4-f22223090dab admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.020685 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24460798-c234-4f92-81f4-f22223090dab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:18.022343 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24460798-c234-4f92-81f4-f22223090dab admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:18.035887 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 789/1571] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:18.035887 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24460798-c234-4f92-81f4-f22223090dab admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:18.042454 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc60e66-e66c-49e2-b06b-a4ef1a79bd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.043065 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc60e66-e66c-49e2-b06b-a4ef1a79bd3e admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.043217 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24460798-c234-4f92-81f4-f22223090dab admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.043314 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc60e66-e66c-49e2-b06b-a4ef1a79bd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.043586 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc60e66-e66c-49e2-b06b-a4ef1a79bd3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.044182 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc60e66-e66c-49e2-b06b-a4ef1a79bd3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:18.046107 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc60e66-e66c-49e2-b06b-a4ef1a79bd3e admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:18.047899 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b602c9ac-0c04-4c85-b48a-695458152f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.048489 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602c9ac-0c04-4c85-b48a-695458152f1b admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.048765 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602c9ac-0c04-4c85-b48a-695458152f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.049034 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602c9ac-0c04-4c85-b48a-695458152f1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.049710 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b602c9ac-0c04-4c85-b48a-695458152f1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:18.051287 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b602c9ac-0c04-4c85-b48a-695458152f1b admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:18.062455 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 790/1572] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:18.063598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b602c9ac-0c04-4c85-b48a-695458152f1b admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:18.073981 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b602c9ac-0c04-4c85-b48a-695458152f1b admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.078739 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 783/1573] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:12:17 2022] POST /identity/v3/projects => generated 391 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:18.078739 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc60e66-e66c-49e2-b06b-a4ef1a79bd3e admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:18.118280 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:18.131802 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167d2a0-93ca-4bd2-9414-3193d99c521b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.132408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167d2a0-93ca-4bd2-9414-3193d99c521b admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.132681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167d2a0-93ca-4bd2-9414-3193d99c521b admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.132956 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167d2a0-93ca-4bd2-9414-3193d99c521b admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.133682 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1167d2a0-93ca-4bd2-9414-3193d99c521b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:18.135576 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1167d2a0-93ca-4bd2-9414-3193d99c521b admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:18.138910 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc60e66-e66c-49e2-b06b-a4ef1a79bd3e admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.143790 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767d0c5-4511-4fbf-b540-d96501766ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.144392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767d0c5-4511-4fbf-b540-d96501766ccc admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.144664 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767d0c5-4511-4fbf-b540-d96501766ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.144928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767d0c5-4511-4fbf-b540-d96501766ccc admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.145584 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3767d0c5-4511-4fbf-b540-d96501766ccc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:18.147506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3767d0c5-4511-4fbf-b540-d96501766ccc admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:18.155914 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:18.167075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 791/1574] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:12:18 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:18.167075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167d2a0-93ca-4bd2-9414-3193d99c521b admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:18.178144 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 784/1575] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:12:18 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:18.179217 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767d0c5-4511-4fbf-b540-d96501766ccc admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:18.229960 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167d2a0-93ca-4bd2-9414-3193d99c521b admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.242836 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767d0c5-4511-4fbf-b540-d96501766ccc admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.247655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc47bf-32ec-40f2-adcb-7313b169e1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.248365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc47bf-32ec-40f2-adcb-7313b169e1be admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.248726 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc47bf-32ec-40f2-adcb-7313b169e1be admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.249061 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc47bf-32ec-40f2-adcb-7313b169e1be admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.249783 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc47bf-32ec-40f2-adcb-7313b169e1be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:18.250869 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: WARNING py.warnings [None req-e8dc47bf-32ec-40f2-adcb-7313b169e1be admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 15:12:18.250869 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: warnings.warn(msg) Sep 11 15:12:18.250869 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: Sep 11 15:12:18.252279 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc47bf-32ec-40f2-adcb-7313b169e1be admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:18.277760 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 785/1576] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:12:18 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:18.278693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc47bf-32ec-40f2-adcb-7313b169e1be admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:18.288991 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc47bf-32ec-40f2-adcb-7313b169e1be admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.293594 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80974c0-535a-4a12-a2a2-67c6f6f8a182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.294187 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80974c0-535a-4a12-a2a2-67c6f6f8a182 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.294458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80974c0-535a-4a12-a2a2-67c6f6f8a182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.294721 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80974c0-535a-4a12-a2a2-67c6f6f8a182 admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.295420 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80974c0-535a-4a12-a2a2-67c6f6f8a182 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:18.297147 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b786b7-7480-44ed-aafc-3aacb2dc57e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.297323 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80974c0-535a-4a12-a2a2-67c6f6f8a182 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:18.297787 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b786b7-7480-44ed-aafc-3aacb2dc57e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.298065 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b786b7-7480-44ed-aafc-3aacb2dc57e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.298336 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b786b7-7480-44ed-aafc-3aacb2dc57e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.298967 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b786b7-7480-44ed-aafc-3aacb2dc57e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:18.300856 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b786b7-7480-44ed-aafc-3aacb2dc57e6 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:18.323984 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 786/1577] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:12:18 2022] POST /identity/v3/users => generated 303 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:18.323984 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80974c0-535a-4a12-a2a2-67c6f6f8a182 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:18.330530 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 792/1578] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:12:18 2022] POST /identity/v3/projects => generated 387 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:18.331411 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b786b7-7480-44ed-aafc-3aacb2dc57e6 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:18.334646 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80974c0-535a-4a12-a2a2-67c6f6f8a182 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.392999 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf847a-d0a0-43a5-9874-cf04e0a0a32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.393661 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b786b7-7480-44ed-aafc-3aacb2dc57e6 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.393876 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf847a-d0a0-43a5-9874-cf04e0a0a32c admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.394218 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf847a-d0a0-43a5-9874-cf04e0a0a32c admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.394551 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf847a-d0a0-43a5-9874-cf04e0a0a32c admin admin] PATH_INFO: `/v3/projects/a9e42cc8e82e41e2a5a33ae922079bc8/users/3eb721f100f24930bafd056d29048561/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.398783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f90e9d-32ae-48fb-b349-a71c2904f672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.399397 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f90e9d-32ae-48fb-b349-a71c2904f672 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.399674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f90e9d-32ae-48fb-b349-a71c2904f672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.399993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f90e9d-32ae-48fb-b349-a71c2904f672 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.400635 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f90e9d-32ae-48fb-b349-a71c2904f672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:18.402285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f90e9d-32ae-48fb-b349-a71c2904f672 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:18.409807 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf847a-d0a0-43a5-9874-cf04e0a0a32c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9e42cc8e82e41e2a5a33ae922079bc8, user_id=3eb721f100f24930bafd056d29048561, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:18.412348 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 793/1579] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:18.412478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf847a-d0a0-43a5-9874-cf04e0a0a32c admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:18.412981 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f90e9d-32ae-48fb-b349-a71c2904f672 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:18.423860 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f90e9d-32ae-48fb-b349-a71c2904f672 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.426937 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 787/1580] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:18 2022] PUT /identity/v3/projects/a9e42cc8e82e41e2a5a33ae922079bc8/users/3eb721f100f24930bafd056d29048561/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:18.427734 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf847a-d0a0-43a5-9874-cf04e0a0a32c admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:18.449160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:18.449845 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:18.484854 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d2a11-98d1-401d-a668-8c2d0e84d841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.485499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d2a11-98d1-401d-a668-8c2d0e84d841 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.485772 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d2a11-98d1-401d-a668-8c2d0e84d841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.486036 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d2a11-98d1-401d-a668-8c2d0e84d841 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.486657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64d2a11-98d1-401d-a668-8c2d0e84d841 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:18.488569 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64d2a11-98d1-401d-a668-8c2d0e84d841 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:18.492039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf847a-d0a0-43a5-9874-cf04e0a0a32c admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.496641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1485c1ce-e7fd-483c-833a-7b3dd1b018dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.497280 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485c1ce-e7fd-483c-833a-7b3dd1b018dc admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.497561 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485c1ce-e7fd-483c-833a-7b3dd1b018dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.497832 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485c1ce-e7fd-483c-833a-7b3dd1b018dc admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.498461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1485c1ce-e7fd-483c-833a-7b3dd1b018dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:18.500445 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1485c1ce-e7fd-483c-833a-7b3dd1b018dc admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:18.511598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 794/1581] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:12:18 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:18.512670 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d2a11-98d1-401d-a668-8c2d0e84d841 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:18.523549 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d2a11-98d1-401d-a668-8c2d0e84d841 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.528993 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 788/1582] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:12:18 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:18.528993 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1485c1ce-e7fd-483c-833a-7b3dd1b018dc admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:18.529835 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f9ab8-64bb-45e8-bd88-c5564bb3d9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.530521 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f9ab8-64bb-45e8-bd88-c5564bb3d9e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.530798 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f9ab8-64bb-45e8-bd88-c5564bb3d9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.531067 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f9ab8-64bb-45e8-bd88-c5564bb3d9e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.531698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023f9ab8-64bb-45e8-bd88-c5564bb3d9e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:18.533655 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023f9ab8-64bb-45e8-bd88-c5564bb3d9e7 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:18.538179 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1485c1ce-e7fd-483c-833a-7b3dd1b018dc admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.543029 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efddf47-b452-416a-8376-407cb4a6c00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.543662 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efddf47-b452-416a-8376-407cb4a6c00f admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.543941 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efddf47-b452-416a-8376-407cb4a6c00f admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.544250 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efddf47-b452-416a-8376-407cb4a6c00f admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.544894 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efddf47-b452-416a-8376-407cb4a6c00f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:18.546595 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efddf47-b452-416a-8376-407cb4a6c00f admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:18.556553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 795/1583] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:12:18 2022] POST /identity/v3/users => generated 292 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:18.560072 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 789/1584] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:18.560227 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f9ab8-64bb-45e8-bd88-c5564bb3d9e7 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:18.560344 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efddf47-b452-416a-8376-407cb4a6c00f admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:18.569587 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f9ab8-64bb-45e8-bd88-c5564bb3d9e7 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.569769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efddf47-b452-416a-8376-407cb4a6c00f admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.574296 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa286dc-1c84-4e47-af45-ebcc4493bd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.574706 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f132b4-b8ce-4bb2-8eda-98c9d217339d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.574902 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa286dc-1c84-4e47-af45-ebcc4493bd1d admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.575175 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa286dc-1c84-4e47-af45-ebcc4493bd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.575307 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f132b4-b8ce-4bb2-8eda-98c9d217339d admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.575439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa286dc-1c84-4e47-af45-ebcc4493bd1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.575583 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f132b4-b8ce-4bb2-8eda-98c9d217339d admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.576221 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f132b4-b8ce-4bb2-8eda-98c9d217339d admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.576444 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa286dc-1c84-4e47-af45-ebcc4493bd1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:18.576839 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f132b4-b8ce-4bb2-8eda-98c9d217339d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:18.578086 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa286dc-1c84-4e47-af45-ebcc4493bd1d admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:18.578775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f132b4-b8ce-4bb2-8eda-98c9d217339d admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:18.589112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 796/1585] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:18.589918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa286dc-1c84-4e47-af45-ebcc4493bd1d admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:18.599716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa286dc-1c84-4e47-af45-ebcc4493bd1d admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.604463 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4caa9-dea7-4906-bfc7-1d1bbce47eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.604770 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 790/1586] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:12:18 2022] POST /identity/v3/users => generated 294 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:18.605103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4caa9-dea7-4906-bfc7-1d1bbce47eef admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.605407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4caa9-dea7-4906-bfc7-1d1bbce47eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.605669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4caa9-dea7-4906-bfc7-1d1bbce47eef admin admin] PATH_INFO: `/v3/projects/7860f1640b1d4d9ebbdff61db2c56d31/users/28c98119f52c4d2ab43ffb539483a823/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.606106 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69ee47-bd2d-4e9f-bd3b-8201353c0f3c None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.606443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69ee47-bd2d-4e9f-bd3b-8201353c0f3c None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.606657 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69ee47-bd2d-4e9f-bd3b-8201353c0f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.621436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4caa9-dea7-4906-bfc7-1d1bbce47eef admin admin] RBAC: Authorizing `identity:create_grant(project_id=7860f1640b1d4d9ebbdff61db2c56d31, user_id=28c98119f52c4d2ab43ffb539483a823, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:18.622464 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: WARNING py.warnings [None req-4ce4caa9-dea7-4906-bfc7-1d1bbce47eef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 15:12:18.622464 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: warnings.warn(msg) Sep 11 15:12:18.622464 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: Sep 11 15:12:18.624406 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4caa9-dea7-4906-bfc7-1d1bbce47eef admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:18.624871 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-0e69ee47-bd2d-4e9f-bd3b-8201353c0f3c None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:18.626085 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-0e69ee47-bd2d-4e9f-bd3b-8201353c0f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:18.634388 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 791/1587] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:18.638351 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69ee47-bd2d-4e9f-bd3b-8201353c0f3c None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:18.639170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 797/1588] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:18 2022] PUT /identity/v3/projects/7860f1640b1d4d9ebbdff61db2c56d31/users/28c98119f52c4d2ab43ffb539483a823/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:18.639981 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4caa9-dea7-4906-bfc7-1d1bbce47eef admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:18.648241 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69ee47-bd2d-4e9f-bd3b-8201353c0f3c None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.715833 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db9b89-5d2c-4135-8f2f-2edf2b5623ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.716128 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4caa9-dea7-4906-bfc7-1d1bbce47eef admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.716446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db9b89-5d2c-4135-8f2f-2edf2b5623ac admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.716718 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db9b89-5d2c-4135-8f2f-2edf2b5623ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.716981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db9b89-5d2c-4135-8f2f-2edf2b5623ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.717647 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db9b89-5d2c-4135-8f2f-2edf2b5623ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:18.719265 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db9b89-5d2c-4135-8f2f-2edf2b5623ac admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:18.720878 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ff704-1771-4169-ad53-01193bba8ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.721573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ff704-1771-4169-ad53-01193bba8ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.721878 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ff704-1771-4169-ad53-01193bba8ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.722149 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ff704-1771-4169-ad53-01193bba8ad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.722790 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1ff704-1771-4169-ad53-01193bba8ad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:18.724407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1ff704-1771-4169-ad53-01193bba8ad5 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:18.732492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 792/1589] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:18.732492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db9b89-5d2c-4135-8f2f-2edf2b5623ac admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:18.737869 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 798/1590] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:18.737869 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ff704-1771-4169-ad53-01193bba8ad5 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:18.742084 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db9b89-5d2c-4135-8f2f-2edf2b5623ac admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.746640 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6401548-3a23-4c21-9588-e3b231c36696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.747230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6401548-3a23-4c21-9588-e3b231c36696 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.747497 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6401548-3a23-4c21-9588-e3b231c36696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.747603 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ff704-1771-4169-ad53-01193bba8ad5 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.747758 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6401548-3a23-4c21-9588-e3b231c36696 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.748386 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6401548-3a23-4c21-9588-e3b231c36696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:18.749999 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6401548-3a23-4c21-9588-e3b231c36696 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:18.753070 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdf8b4-d0df-45b3-9a7d-3bc9b53da312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.753738 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdf8b4-d0df-45b3-9a7d-3bc9b53da312 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.754018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdf8b4-d0df-45b3-9a7d-3bc9b53da312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.754356 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdf8b4-d0df-45b3-9a7d-3bc9b53da312 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.754993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfdf8b4-d0df-45b3-9a7d-3bc9b53da312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:18.756774 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfdf8b4-d0df-45b3-9a7d-3bc9b53da312 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:18.760222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 793/1591] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:18.760982 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6401548-3a23-4c21-9588-e3b231c36696 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:18.770427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 799/1592] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:18.770427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdf8b4-d0df-45b3-9a7d-3bc9b53da312 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:18.770709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6401548-3a23-4c21-9588-e3b231c36696 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.775489 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88039e-17ef-405c-9f4a-ec442a2f5e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.776083 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88039e-17ef-405c-9f4a-ec442a2f5e62 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.776355 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88039e-17ef-405c-9f4a-ec442a2f5e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.776620 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88039e-17ef-405c-9f4a-ec442a2f5e62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.777280 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be88039e-17ef-405c-9f4a-ec442a2f5e62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:18.778983 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be88039e-17ef-405c-9f4a-ec442a2f5e62 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:18.780132 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdf8b4-d0df-45b3-9a7d-3bc9b53da312 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.784866 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3855564-4902-44de-b51e-e63804f295fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.785507 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3855564-4902-44de-b51e-e63804f295fa admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.785786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3855564-4902-44de-b51e-e63804f295fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.786059 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3855564-4902-44de-b51e-e63804f295fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.786730 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3855564-4902-44de-b51e-e63804f295fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:18.788308 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3855564-4902-44de-b51e-e63804f295fa admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:18.790475 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 794/1593] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:18.791773 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f78f7a-a7dd-4a74-b05f-0631c4bef0a6 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.792058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f78f7a-a7dd-4a74-b05f-0631c4bef0a6 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.792356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f78f7a-a7dd-4a74-b05f-0631c4bef0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.799766 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 800/1594] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:18.801050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3855564-4902-44de-b51e-e63804f295fa admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:18.822537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-83f78f7a-a7dd-4a74-b05f-0631c4bef0a6 None None] MFA Rules not processed for user `28c98119f52c4d2ab43ffb539483a823`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:18.823963 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-83f78f7a-a7dd-4a74-b05f-0631c4bef0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:18.843008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:18.858609 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3855564-4902-44de-b51e-e63804f295fa admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.862406 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd39bd59-9bbf-47fe-9671-16cd058c65bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.863033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39bd59-9bbf-47fe-9671-16cd058c65bd service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.863303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39bd59-9bbf-47fe-9671-16cd058c65bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.863552 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39bd59-9bbf-47fe-9671-16cd058c65bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.866300 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:18.873301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd39bd59-9bbf-47fe-9671-16cd058c65bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:18.874731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd39bd59-9bbf-47fe-9671-16cd058c65bd service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:18.884307 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 801/1595] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:18.885094 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd39bd59-9bbf-47fe-9671-16cd058c65bd service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:18.892553 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 795/1596] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:18.894337 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd39bd59-9bbf-47fe-9671-16cd058c65bd service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.897708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f78f7a-a7dd-4a74-b05f-0631c4bef0a6 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:18.900566 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10417af0-5bc1-4ccb-bf37-d0bb0d9603cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.901355 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10417af0-5bc1-4ccb-bf37-d0bb0d9603cc admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.901706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10417af0-5bc1-4ccb-bf37-d0bb0d9603cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.902011 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10417af0-5bc1-4ccb-bf37-d0bb0d9603cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.902623 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10417af0-5bc1-4ccb-bf37-d0bb0d9603cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:18.904320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10417af0-5bc1-4ccb-bf37-d0bb0d9603cc admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:18.910265 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f78f7a-a7dd-4a74-b05f-0631c4bef0a6 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.915430 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf07d5-b82b-4068-97ed-290b1e77957a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.916051 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf07d5-b82b-4068-97ed-290b1e77957a admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.919477 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf07d5-b82b-4068-97ed-290b1e77957a admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.919712 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 802/1597] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:18.919797 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf07d5-b82b-4068-97ed-290b1e77957a admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.919797 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf07d5-b82b-4068-97ed-290b1e77957a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:18.919957 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10417af0-5bc1-4ccb-bf37-d0bb0d9603cc admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:18.920044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf07d5-b82b-4068-97ed-290b1e77957a admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:18.933044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 796/1598] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:18.933044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf07d5-b82b-4068-97ed-290b1e77957a admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:18.933345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10417af0-5bc1-4ccb-bf37-d0bb0d9603cc admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.936591 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37fe287-7ec3-4300-a08a-d70d734f29a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.937317 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37fe287-7ec3-4300-a08a-d70d734f29a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.937652 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37fe287-7ec3-4300-a08a-d70d734f29a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.938012 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37fe287-7ec3-4300-a08a-d70d734f29a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.938808 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37fe287-7ec3-4300-a08a-d70d734f29a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:18.940575 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37fe287-7ec3-4300-a08a-d70d734f29a8 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:18.944330 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf07d5-b82b-4068-97ed-290b1e77957a admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.949647 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e2c0cd-7d6e-45d0-8b24-9246ed5f28f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.950559 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e2c0cd-7d6e-45d0-8b24-9246ed5f28f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.950559 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e2c0cd-7d6e-45d0-8b24-9246ed5f28f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.951254 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e2c0cd-7d6e-45d0-8b24-9246ed5f28f6 admin admin] PATH_INFO: `/v3/projects/3a9a84610b96408189df339505ac4e2d/users/8d5fa4dea79740708b0d897e70f7b47f/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.952668 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 803/1599] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:18.954215 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37fe287-7ec3-4300-a08a-d70d734f29a8 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:18.966059 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37fe287-7ec3-4300-a08a-d70d734f29a8 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:18.967724 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e2c0cd-7d6e-45d0-8b24-9246ed5f28f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a9a84610b96408189df339505ac4e2d, user_id=8d5fa4dea79740708b0d897e70f7b47f, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:18.970605 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e2c0cd-7d6e-45d0-8b24-9246ed5f28f6 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:18.971524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bd277-66d2-465b-b8a4-b3d8169d939b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:18.972234 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bd277-66d2-465b-b8a4-b3d8169d939b admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:18.972576 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bd277-66d2-465b-b8a4-b3d8169d939b admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:18.972851 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bd277-66d2-465b-b8a4-b3d8169d939b admin admin] PATH_INFO: `/v3/projects/ae744511b5684cbcb0dc51495d12e666/users/d4a88fb1c9cd42c498e4c71b31a50ae5/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:18.986608 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 797/1600] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:18 2022] PUT /identity/v3/projects/3a9a84610b96408189df339505ac4e2d/users/8d5fa4dea79740708b0d897e70f7b47f/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:18.987619 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e2c0cd-7d6e-45d0-8b24-9246ed5f28f6 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:18.998892 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5bd277-66d2-465b-b8a4-b3d8169d939b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae744511b5684cbcb0dc51495d12e666, user_id=d4a88fb1c9cd42c498e4c71b31a50ae5, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:19.001610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5bd277-66d2-465b-b8a4-b3d8169d939b admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:19.023434 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 804/1601] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:18 2022] PUT /identity/v3/projects/ae744511b5684cbcb0dc51495d12e666/users/d4a88fb1c9cd42c498e4c71b31a50ae5/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:19.025302 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bd277-66d2-465b-b8a4-b3d8169d939b admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:19.059858 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e2c0cd-7d6e-45d0-8b24-9246ed5f28f6 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:19.099480 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bd277-66d2-465b-b8a4-b3d8169d939b admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:19.105333 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442acb8-7e54-47c1-b946-baf094a6f94d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:19.106137 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442acb8-7e54-47c1-b946-baf094a6f94d service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:19.106633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442acb8-7e54-47c1-b946-baf094a6f94d service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:19.107302 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442acb8-7e54-47c1-b946-baf094a6f94d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:19.152278 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63717c-2598-47a4-9731-f95d78af9c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:19.152891 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63717c-2598-47a4-9731-f95d78af9c43 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:19.153518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63717c-2598-47a4-9731-f95d78af9c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:19.153518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63717c-2598-47a4-9731-f95d78af9c43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:19.154449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63717c-2598-47a4-9731-f95d78af9c43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:19.156425 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63717c-2598-47a4-9731-f95d78af9c43 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:19.167020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 798/1602] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:19.167656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63717c-2598-47a4-9731-f95d78af9c43 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:19.176416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2442acb8-7e54-47c1-b946-baf094a6f94d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:19.177923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2442acb8-7e54-47c1-b946-baf094a6f94d service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:19.179046 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63717c-2598-47a4-9731-f95d78af9c43 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:19.185553 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675ed8a-f30e-4261-a61a-8205c35d22c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:19.186316 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675ed8a-f30e-4261-a61a-8205c35d22c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:19.186536 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675ed8a-f30e-4261-a61a-8205c35d22c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:19.186807 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675ed8a-f30e-4261-a61a-8205c35d22c2 admin admin] PATH_INFO: `/v3/projects/7933806a47704d2d867be7312890b538/users/2bd47bf9aa4142b1b7ec8d02e8661578/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:19.188078 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 805/1603] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:19.189017 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442acb8-7e54-47c1-b946-baf094a6f94d service cinder] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:19.204449 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442acb8-7e54-47c1-b946-baf094a6f94d service cinder] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:19.210358 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4675ed8a-f30e-4261-a61a-8205c35d22c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7933806a47704d2d867be7312890b538, user_id=2bd47bf9aa4142b1b7ec8d02e8661578, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:19.210544 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db35c29-b72b-46b0-b3bc-24d77e6812e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:19.211854 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db35c29-b72b-46b0-b3bc-24d77e6812e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:19.212141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db35c29-b72b-46b0-b3bc-24d77e6812e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:19.212419 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db35c29-b72b-46b0-b3bc-24d77e6812e7 admin admin] PATH_INFO: `/v3/projects/053104c5c4504dba86cd8749650f94b9/users/1121f5db549842a8b1b9b3bbaf0cfb42/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:19.215390 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4675ed8a-f30e-4261-a61a-8205c35d22c2 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:19.230521 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db35c29-b72b-46b0-b3bc-24d77e6812e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=053104c5c4504dba86cd8749650f94b9, user_id=1121f5db549842a8b1b9b3bbaf0cfb42, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:19.233509 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db35c29-b72b-46b0-b3bc-24d77e6812e7 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:19.234828 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 799/1604] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:19 2022] PUT /identity/v3/projects/7933806a47704d2d867be7312890b538/users/2bd47bf9aa4142b1b7ec8d02e8661578/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:19.235774 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675ed8a-f30e-4261-a61a-8205c35d22c2 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:19.261192 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 806/1605] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:19 2022] PUT /identity/v3/projects/053104c5c4504dba86cd8749650f94b9/users/1121f5db549842a8b1b9b3bbaf0cfb42/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:19.262588 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd2f6a-5510-47ea-9b03-692336434904 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:19.262941 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd2f6a-5510-47ea-9b03-692336434904 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:19.263283 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd2f6a-5510-47ea-9b03-692336434904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:19.290056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:19.307073 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-d4cd2f6a-5510-47ea-9b03-692336434904 None None] MFA Rules not processed for user `8d5fa4dea79740708b0d897e70f7b47f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:19.308231 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-d4cd2f6a-5510-47ea-9b03-692336434904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:19.337075 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675ed8a-f30e-4261-a61a-8205c35d22c2 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:19.352830 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:19.416517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c813357-d5f8-4a3e-951c-1828a0f9fdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:19.417391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c813357-d5f8-4a3e-951c-1828a0f9fdab admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:19.417671 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c813357-d5f8-4a3e-951c-1828a0f9fdab admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:19.418081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c813357-d5f8-4a3e-951c-1828a0f9fdab admin admin] PATH_INFO: `/v3/projects/c0693786c6694f9e8a8bb8bec8fba0b5/users/c44105f2d90e469fb03f311a4cf1476f/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:19.436961 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c813357-d5f8-4a3e-951c-1828a0f9fdab admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0693786c6694f9e8a8bb8bec8fba0b5, user_id=c44105f2d90e469fb03f311a4cf1476f, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:19.441123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c813357-d5f8-4a3e-951c-1828a0f9fdab admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:19.461892 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 800/1606] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:19 2022] PUT /identity/v3/projects/c0693786c6694f9e8a8bb8bec8fba0b5/users/c44105f2d90e469fb03f311a4cf1476f/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:19.463386 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03515f07-e148-4258-a68b-0ec8fdb1e235 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:19.463758 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03515f07-e148-4258-a68b-0ec8fdb1e235 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:19.464118 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03515f07-e148-4258-a68b-0ec8fdb1e235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:19.471035 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 807/1607] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:19.471035 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd2f6a-5510-47ea-9b03-692336434904 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:19.497628 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-03515f07-e148-4258-a68b-0ec8fdb1e235 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:19.499188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-03515f07-e148-4258-a68b-0ec8fdb1e235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:19.500435 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd2f6a-5510-47ea-9b03-692336434904 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:19.585522 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 801/1608] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:19.587428 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7db7a3-9713-4264-8e1c-47498d662d9b None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:19.587710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7db7a3-9713-4264-8e1c-47498d662d9b None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:19.587979 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7db7a3-9713-4264-8e1c-47498d662d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:19.598593 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e0e1a-3190-46df-bfeb-8c1dee8c16af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:19.599713 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e0e1a-3190-46df-bfeb-8c1dee8c16af admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:19.600343 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e0e1a-3190-46df-bfeb-8c1dee8c16af admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:19.600859 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e0e1a-3190-46df-bfeb-8c1dee8c16af admin admin] PATH_INFO: `/v3/projects/2273d07b98054e3987568fc043597b78/users/5e4f8b13b66449cf849ff9cb73938c4c/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:19.609545 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-4a7db7a3-9713-4264-8e1c-47498d662d9b None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:19.610967 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-4a7db7a3-9713-4264-8e1c-47498d662d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:19.617677 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e0e1a-3190-46df-bfeb-8c1dee8c16af admin admin] RBAC: Authorizing `identity:create_grant(project_id=2273d07b98054e3987568fc043597b78, user_id=5e4f8b13b66449cf849ff9cb73938c4c, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:19.622723 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 802/1609] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:19.622903 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e0e1a-3190-46df-bfeb-8c1dee8c16af admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:19.623006 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a64e21-f0ec-4a75-b80b-59769adc461c None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:19.623006 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a64e21-f0ec-4a75-b80b-59769adc461c None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:19.623006 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a64e21-f0ec-4a75-b80b-59769adc461c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:19.644952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 808/1610] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:19 2022] PUT /identity/v3/projects/2273d07b98054e3987568fc043597b78/users/5e4f8b13b66449cf849ff9cb73938c4c/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:19.651014 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720a7e7-5672-4fbd-afd6-1058fa99ee2c None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:19.651457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720a7e7-5672-4fbd-afd6-1058fa99ee2c None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:19.651824 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720a7e7-5672-4fbd-afd6-1058fa99ee2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:19.655340 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-38a64e21-f0ec-4a75-b80b-59769adc461c None None] MFA Rules not processed for user `1121f5db549842a8b1b9b3bbaf0cfb42`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:19.656631 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-38a64e21-f0ec-4a75-b80b-59769adc461c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:19.673741 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-c720a7e7-5672-4fbd-afd6-1058fa99ee2c None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:19.675472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-c720a7e7-5672-4fbd-afd6-1058fa99ee2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:19.702823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:19.762519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 803/1611] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:19.762519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a14e3-3a7e-4a58-86c5-847ea7e2dcfc None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:19.762519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a14e3-3a7e-4a58-86c5-847ea7e2dcfc None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:19.763196 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a14e3-3a7e-4a58-86c5-847ea7e2dcfc None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:19.765782 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 804/1612] 10.222.0.44 () {54 vars in 962 bytes} [Sun Sep 11 15:12:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:12:19.775841 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a14e3-3a7e-4a58-86c5-847ea7e2dcfc None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:19.808262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 809/1613] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:19.808587 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c720a7e7-5672-4fbd-afd6-1058fa99ee2c None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:19.924523 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:19.949150 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a14e3-3a7e-4a58-86c5-847ea7e2dcfc None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:19.954972 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c720a7e7-5672-4fbd-afd6-1058fa99ee2c None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:19.956909 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f5c71-5875-458b-b14c-d987bc8933aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:19.958343 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f5c71-5875-458b-b14c-d987bc8933aa service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:19.958430 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f5c71-5875-458b-b14c-d987bc8933aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:19.958897 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f5c71-5875-458b-b14c-d987bc8933aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:19.965253 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86349d7-a515-4fdb-9a10-e01bd83390e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:19.968445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86349d7-a515-4fdb-9a10-e01bd83390e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:19.968976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86349d7-a515-4fdb-9a10-e01bd83390e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:19.969453 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86349d7-a515-4fdb-9a10-e01bd83390e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:19.978940 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f5c71-5875-458b-b14c-d987bc8933aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:19.982259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f5c71-5875-458b-b14c-d987bc8933aa service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:19.990103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86349d7-a515-4fdb-9a10-e01bd83390e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:19.993850 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86349d7-a515-4fdb-9a10-e01bd83390e7 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:20.007667 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 805/1614] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:20.011810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842c318-4e5e-40e2-9de8-cac9547bf242 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:20.013617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842c318-4e5e-40e2-9de8-cac9547bf242 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:20.016368 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842c318-4e5e-40e2-9de8-cac9547bf242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:20.036911 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 810/1615] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:20.038256 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c263c1f-73a7-4b21-b9ba-5f3de06f1c8e None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:20.038539 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c263c1f-73a7-4b21-b9ba-5f3de06f1c8e None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:20.038807 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c263c1f-73a7-4b21-b9ba-5f3de06f1c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:20.085295 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-3c263c1f-73a7-4b21-b9ba-5f3de06f1c8e None None] MFA Rules not processed for user `453297dac64046d19fc1c331d2d85abc`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:20.086505 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-3c263c1f-73a7-4b21-b9ba-5f3de06f1c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:20.102130 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-5842c318-4e5e-40e2-9de8-cac9547bf242 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:20.104205 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-5842c318-4e5e-40e2-9de8-cac9547bf242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:20.136359 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 806/1616] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:20.138222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842c318-4e5e-40e2-9de8-cac9547bf242 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:20.168238 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842c318-4e5e-40e2-9de8-cac9547bf242 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:20.174818 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 811/1617] 10.222.0.44 () {58 vars in 1059 bytes} [Sun Sep 11 15:12:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:20.176777 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9213e-628a-46c2-9f0d-1b58bdbeacc3 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:20.177746 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9213e-628a-46c2-9f0d-1b58bdbeacc3 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:20.177746 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9213e-628a-46c2-9f0d-1b58bdbeacc3 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:20.179301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 812/1618] 10.222.0.44 () {54 vars in 962 bytes} [Sun Sep 11 15:12:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:12:20.180364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9213e-628a-46c2-9f0d-1b58bdbeacc3 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:20.181942 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14540b74-34d5-4e9e-b3ac-eb713d8918b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:20.183466 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14540b74-34d5-4e9e-b3ac-eb713d8918b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:20.184372 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14540b74-34d5-4e9e-b3ac-eb713d8918b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:20.185212 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14540b74-34d5-4e9e-b3ac-eb713d8918b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:20.193216 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9213e-628a-46c2-9f0d-1b58bdbeacc3 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:20.197533 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc1346e-63da-41ed-bf9b-a65808a513c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:20.198251 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1346e-63da-41ed-bf9b-a65808a513c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:20.198627 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1346e-63da-41ed-bf9b-a65808a513c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:20.199122 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1346e-63da-41ed-bf9b-a65808a513c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:20.216763 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14540b74-34d5-4e9e-b3ac-eb713d8918b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:20.221067 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14540b74-34d5-4e9e-b3ac-eb713d8918b7 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:20.222699 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc1346e-63da-41ed-bf9b-a65808a513c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:20.224162 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc1346e-63da-41ed-bf9b-a65808a513c2 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:20.245431 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 807/1619] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:20.247082 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14540b74-34d5-4e9e-b3ac-eb713d8918b7 service neutron] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:20.251641 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 813/1620] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:20.251933 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786fe8dd-d8b4-4a1c-b94a-0412e18c2b3a None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:20.252218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786fe8dd-d8b4-4a1c-b94a-0412e18c2b3a None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:20.254223 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786fe8dd-d8b4-4a1c-b94a-0412e18c2b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:20.275781 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14540b74-34d5-4e9e-b3ac-eb713d8918b7 service neutron] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:20.289696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb6ab08-e448-4b50-85b0-2351d10459a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:20.290929 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6ab08-e448-4b50-85b0-2351d10459a8 service nova] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:20.291442 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6ab08-e448-4b50-85b0-2351d10459a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:20.291915 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6ab08-e448-4b50-85b0-2351d10459a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:20.293352 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-786fe8dd-d8b4-4a1c-b94a-0412e18c2b3a None None] MFA Rules not processed for user `453297dac64046d19fc1c331d2d85abc`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:20.295037 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-786fe8dd-d8b4-4a1c-b94a-0412e18c2b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:20.315481 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 814/1621] 10.222.0.44 () {58 vars in 1059 bytes} [Sun Sep 11 15:12:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:20.317174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786fe8dd-d8b4-4a1c-b94a-0412e18c2b3a None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:20.329565 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786fe8dd-d8b4-4a1c-b94a-0412e18c2b3a None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:20.337314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0da6a-edfb-41ed-aac1-351ac4e352f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:20.338546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0da6a-edfb-41ed-aac1-351ac4e352f7 service nova] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:20.339060 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0da6a-edfb-41ed-aac1-351ac4e352f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:20.339529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0da6a-edfb-41ed-aac1-351ac4e352f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:20.350695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d0da6a-edfb-41ed-aac1-351ac4e352f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:20.353026 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d0da6a-edfb-41ed-aac1-351ac4e352f7 service nova] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:20.376487 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 815/1622] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:20.416199 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb6ab08-e448-4b50-85b0-2351d10459a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:20.420006 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb6ab08-e448-4b50-85b0-2351d10459a8 service nova] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:20.439763 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 808/1623] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:20.675060 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5a190-e3af-45d6-a6f2-342fb23735d0 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:20.675367 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5a190-e3af-45d6-a6f2-342fb23735d0 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:20.675677 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5a190-e3af-45d6-a6f2-342fb23735d0 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:20.677309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 816/1624] 10.222.0.44 () {54 vars in 912 bytes} [Sun Sep 11 15:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 15:12:20.683034 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd36f0-f3e5-49ed-b96c-baee671af788 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:20.683398 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd36f0-f3e5-49ed-b96c-baee671af788 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:20.683644 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd36f0-f3e5-49ed-b96c-baee671af788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:20.706384 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-6abd36f0-f3e5-49ed-b96c-baee671af788 None None] MFA Rules not processed for user `453297dac64046d19fc1c331d2d85abc`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:20.707588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-6abd36f0-f3e5-49ed-b96c-baee671af788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:20.725544 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 809/1625] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 15:12:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:12:20.725886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c0309-f49b-4d81-931e-bd8936cf5c5a None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:20.726191 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c0309-f49b-4d81-931e-bd8936cf5c5a None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:20.726459 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c0309-f49b-4d81-931e-bd8936cf5c5a None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:20.727900 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 817/1626] 10.222.0.44 () {54 vars in 912 bytes} [Sun Sep 11 15:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 15:12:20.741954 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c0309-f49b-4d81-931e-bd8936cf5c5a None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:20.750495 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e0e1a-cad8-45ab-9a0b-a738ebe0ea4e None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:20.750792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e0e1a-cad8-45ab-9a0b-a738ebe0ea4e None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:20.751096 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e0e1a-cad8-45ab-9a0b-a738ebe0ea4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:20.798663 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-9e9e0e1a-cad8-45ab-9a0b-a738ebe0ea4e None None] MFA Rules not processed for user `453297dac64046d19fc1c331d2d85abc`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:20.802237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-9e9e0e1a-cad8-45ab-9a0b-a738ebe0ea4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:20.827865 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 810/1627] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 15:12:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:20.841761 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e0e1a-cad8-45ab-9a0b-a738ebe0ea4e None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:20.869088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c0309-f49b-4d81-931e-bd8936cf5c5a None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:20.873295 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85454929-b22a-460e-82cb-a05663c3da3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:20.874414 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85454929-b22a-460e-82cb-a05663c3da3e service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:20.874747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85454929-b22a-460e-82cb-a05663c3da3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:20.875092 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85454929-b22a-460e-82cb-a05663c3da3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:20.890503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85454929-b22a-460e-82cb-a05663c3da3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:20.892039 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85454929-b22a-460e-82cb-a05663c3da3e service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:20.906728 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 818/1628] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:12:20.912769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:20.923692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e0e1a-cad8-45ab-9a0b-a738ebe0ea4e None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:20.927959 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f513d-30cf-4b79-94ef-33dbcb147a9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:20.928640 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f513d-30cf-4b79-94ef-33dbcb147a9e service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:20.928981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f513d-30cf-4b79-94ef-33dbcb147a9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:20.929428 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f513d-30cf-4b79-94ef-33dbcb147a9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:20.940923 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f513d-30cf-4b79-94ef-33dbcb147a9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:20.942490 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f513d-30cf-4b79-94ef-33dbcb147a9e service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:20.958650 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 811/1629] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:12:21.841359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85454929-b22a-460e-82cb-a05663c3da3e service cinder] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:21.857083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85454929-b22a-460e-82cb-a05663c3da3e service cinder] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:21.865731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ffb6c3-5c3e-4cd2-ae41-69d6fc42dd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:21.868900 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffb6c3-5c3e-4cd2-ae41-69d6fc42dd88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:21.870906 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffb6c3-5c3e-4cd2-ae41-69d6fc42dd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:21.871867 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffb6c3-5c3e-4cd2-ae41-69d6fc42dd88 admin admin] PATH_INFO: `/v3/users/1121f5db549842a8b1b9b3bbaf0cfb42` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:21.873780 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ffb6c3-5c3e-4cd2-ae41-69d6fc42dd88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1121f5db549842a8b1b9b3bbaf0cfb42)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:21.875349 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: WARNING py.warnings [None req-76ffb6c3-5c3e-4cd2-ae41-69d6fc42dd88 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 15:12:21.875349 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: warnings.warn(msg) Sep 11 15:12:21.875349 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: Sep 11 15:12:21.877666 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ffb6c3-5c3e-4cd2-ae41-69d6fc42dd88 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:21.930429 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:22.009457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-76ffb6c3-5c3e-4cd2-ae41-69d6fc42dd88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1121f5db549842a8b1b9b3bbaf0cfb42'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:22.023442 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-76ffb6c3-5c3e-4cd2-ae41-69d6fc42dd88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1121f5db549842a8b1b9b3bbaf0cfb42'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:22.082973 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-76ffb6c3-5c3e-4cd2-ae41-69d6fc42dd88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1121f5db549842a8b1b9b3bbaf0cfb42'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:22.083696 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-76ffb6c3-5c3e-4cd2-ae41-69d6fc42dd88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1121f5db549842a8b1b9b3bbaf0cfb42'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:22.114691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-76ffb6c3-5c3e-4cd2-ae41-69d6fc42dd88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1121f5db549842a8b1b9b3bbaf0cfb42'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:22.128206 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 819/1630] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:12:21 2022] DELETE /identity/v3/users/1121f5db549842a8b1b9b3bbaf0cfb42 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:22.130122 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7fc3c-4185-4325-bc2c-99ed7050d100 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:22.130972 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7fc3c-4185-4325-bc2c-99ed7050d100 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:22.131462 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7fc3c-4185-4325-bc2c-99ed7050d100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:22.167656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-26f7fc3c-4185-4325-bc2c-99ed7050d100 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:22.170134 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-26f7fc3c-4185-4325-bc2c-99ed7050d100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:22.265303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 812/1631] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:22.298794 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ffb6c3-5c3e-4cd2-ae41-69d6fc42dd88 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:22.300501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-76ffb6c3-5c3e-4cd2-ae41-69d6fc42dd88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:22.387270 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ffb6c3-5c3e-4cd2-ae41-69d6fc42dd88 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:22.394038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bba35b-9dfc-41e4-9e6f-612c978394c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:22.394685 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bba35b-9dfc-41e4-9e6f-612c978394c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:22.395346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bba35b-9dfc-41e4-9e6f-612c978394c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:22.395346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bba35b-9dfc-41e4-9e6f-612c978394c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:22.416055 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bba35b-9dfc-41e4-9e6f-612c978394c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:22.417775 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bba35b-9dfc-41e4-9e6f-612c978394c3 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:22.430639 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 820/1632] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:24.352339 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7fc3c-4185-4325-bc2c-99ed7050d100 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:24.354487 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-26f7fc3c-4185-4325-bc2c-99ed7050d100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:24.370927 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7fc3c-4185-4325-bc2c-99ed7050d100 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:24.383429 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca226907-2448-4181-a346-5de0fbb69ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:24.384152 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca226907-2448-4181-a346-5de0fbb69ccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:24.386322 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca226907-2448-4181-a346-5de0fbb69ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:24.386612 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca226907-2448-4181-a346-5de0fbb69ccd admin admin] PATH_INFO: `/v3/projects/053104c5c4504dba86cd8749650f94b9` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:24.388334 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca226907-2448-4181-a346-5de0fbb69ccd admin admin] RBAC: Authorizing `identity:delete_project(project_id=053104c5c4504dba86cd8749650f94b9)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:24.390543 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: WARNING py.warnings [None req-ca226907-2448-4181-a346-5de0fbb69ccd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 15:12:24.390543 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: warnings.warn(msg) Sep 11 15:12:24.390543 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: Sep 11 15:12:24.392488 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca226907-2448-4181-a346-5de0fbb69ccd admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:24.538943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-ca226907-2448-4181-a346-5de0fbb69ccd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '053104c5c4504dba86cd8749650f94b9', 'request_id': 'req-ca226907-2448-4181-a346-5de0fbb69ccd'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:24.557927 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-ca226907-2448-4181-a346-5de0fbb69ccd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '053104c5c4504dba86cd8749650f94b9', 'request_id': 'req-ca226907-2448-4181-a346-5de0fbb69ccd'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:24.581515 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-ca226907-2448-4181-a346-5de0fbb69ccd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '053104c5c4504dba86cd8749650f94b9', 'request_id': 'req-ca226907-2448-4181-a346-5de0fbb69ccd'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:24.648273 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-ca226907-2448-4181-a346-5de0fbb69ccd admin admin] The token cache is being invalidate because project 053104c5c4504dba86cd8749650f94b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:12:24.653413 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-ca226907-2448-4181-a346-5de0fbb69ccd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:24.654662 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-ca226907-2448-4181-a346-5de0fbb69ccd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:24.656951 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 813/1633] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:12:24 2022] DELETE /identity/v3/projects/053104c5c4504dba86cd8749650f94b9 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:24.661902 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a4928-5a76-49d4-b9f6-76aa6bf74179 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:24.663527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a4928-5a76-49d4-b9f6-76aa6bf74179 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:24.664001 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a4928-5a76-49d4-b9f6-76aa6bf74179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:24.706894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-305a4928-5a76-49d4-b9f6-76aa6bf74179 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:24.709106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-305a4928-5a76-49d4-b9f6-76aa6bf74179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:24.721809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7a5f3-fd94-4674-b2be-ff6ec1c5234d None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:24.722429 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7a5f3-fd94-4674-b2be-ff6ec1c5234d None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:24.723564 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7a5f3-fd94-4674-b2be-ff6ec1c5234d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:24.845037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-23a7a5f3-fd94-4674-b2be-ff6ec1c5234d None None] MFA Rules not processed for user `d4a88fb1c9cd42c498e4c71b31a50ae5`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:24.846372 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-23a7a5f3-fd94-4674-b2be-ff6ec1c5234d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:24.910233 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 821/1634] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:12:24.911123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a4928-5a76-49d4-b9f6-76aa6bf74179 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:24.912528 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-305a4928-5a76-49d4-b9f6-76aa6bf74179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:24.926588 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a4928-5a76-49d4-b9f6-76aa6bf74179 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:24.935375 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf5459-703e-4c6e-aa48-bba2acd844d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:24.935698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf5459-703e-4c6e-aa48-bba2acd844d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:24.935799 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf5459-703e-4c6e-aa48-bba2acd844d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:24.936011 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf5459-703e-4c6e-aa48-bba2acd844d7 admin admin] PATH_INFO: `/v3/users/8d5fa4dea79740708b0d897e70f7b47f` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:24.937996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf5459-703e-4c6e-aa48-bba2acd844d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d5fa4dea79740708b0d897e70f7b47f)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:24.942273 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf5459-703e-4c6e-aa48-bba2acd844d7 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:24.983911 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:24.990388 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 814/1635] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:24 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:24.993949 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450eaed-153c-40fe-bf1a-7cd1230f5101 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:24.994490 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450eaed-153c-40fe-bf1a-7cd1230f5101 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:24.995013 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450eaed-153c-40fe-bf1a-7cd1230f5101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:25.036478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-2450eaed-153c-40fe-bf1a-7cd1230f5101 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:25.039956 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-2450eaed-153c-40fe-bf1a-7cd1230f5101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:25.041547 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:25.057375 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-41cf5459-703e-4c6e-aa48-bba2acd844d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d5fa4dea79740708b0d897e70f7b47f'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:25.060468 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 815/1636] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:25.061563 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2450eaed-153c-40fe-bf1a-7cd1230f5101 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:25.063208 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-2450eaed-153c-40fe-bf1a-7cd1230f5101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:25.085015 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2450eaed-153c-40fe-bf1a-7cd1230f5101 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:25.091691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-41cf5459-703e-4c6e-aa48-bba2acd844d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d5fa4dea79740708b0d897e70f7b47f'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:25.097555 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0f2f1-65a9-4420-aa71-02ff84985e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:25.098578 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0f2f1-65a9-4420-aa71-02ff84985e54 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:25.099328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0f2f1-65a9-4420-aa71-02ff84985e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:25.100684 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0f2f1-65a9-4420-aa71-02ff84985e54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:25.102437 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf0f2f1-65a9-4420-aa71-02ff84985e54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:25.106121 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf0f2f1-65a9-4420-aa71-02ff84985e54 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:25.119744 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-41cf5459-703e-4c6e-aa48-bba2acd844d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d5fa4dea79740708b0d897e70f7b47f'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:25.121293 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-41cf5459-703e-4c6e-aa48-bba2acd844d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d5fa4dea79740708b0d897e70f7b47f'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:25.137985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-41cf5459-703e-4c6e-aa48-bba2acd844d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d5fa4dea79740708b0d897e70f7b47f'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:25.158627 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 822/1637] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:12:24 2022] DELETE /identity/v3/users/8d5fa4dea79740708b0d897e70f7b47f => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:25.161317 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf5459-703e-4c6e-aa48-bba2acd844d7 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:25.168899 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-41cf5459-703e-4c6e-aa48-bba2acd844d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:25.216757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 816/1638] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:12:25 2022] POST /identity/v3/projects => generated 403 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:25.219516 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b002f62-d457-433b-8da5-3e671fff7ba2 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:25.222165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b002f62-d457-433b-8da5-3e671fff7ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:25.222558 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b002f62-d457-433b-8da5-3e671fff7ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:25.271012 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-7b002f62-d457-433b-8da5-3e671fff7ba2 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:25.272664 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-7b002f62-d457-433b-8da5-3e671fff7ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:25.340028 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf5459-703e-4c6e-aa48-bba2acd844d7 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:25.468611 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 817/1639] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:25.469824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b002f62-d457-433b-8da5-3e671fff7ba2 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:25.474003 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-7b002f62-d457-433b-8da5-3e671fff7ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:25.506078 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2bc4fd-2404-4103-81a7-6d4d81418d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:25.506631 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bc4fd-2404-4103-81a7-6d4d81418d34 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:25.506898 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bc4fd-2404-4103-81a7-6d4d81418d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:25.507173 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bc4fd-2404-4103-81a7-6d4d81418d34 admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:25.507871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2bc4fd-2404-4103-81a7-6d4d81418d34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:12:25.508185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2bc4fd-2404-4103-81a7-6d4d81418d34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:25.510187 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2bc4fd-2404-4103-81a7-6d4d81418d34 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:25.514592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b002f62-d457-433b-8da5-3e671fff7ba2 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:25.529474 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacdc81-8a99-4de3-a9bf-286ec76d30d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:25.530369 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacdc81-8a99-4de3-a9bf-286ec76d30d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:25.530856 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacdc81-8a99-4de3-a9bf-286ec76d30d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:25.531137 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacdc81-8a99-4de3-a9bf-286ec76d30d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:25.532560 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcacdc81-8a99-4de3-a9bf-286ec76d30d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:25.536318 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 823/1640] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:12:25.542805 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2bc4fd-2404-4103-81a7-6d4d81418d34 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:25.563089 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2bc4fd-2404-4103-81a7-6d4d81418d34 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:25.565113 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcacdc81-8a99-4de3-a9bf-286ec76d30d7 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:25.573674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbdb4a-5b13-4ed2-a09a-b02779b87a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:25.574311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbdb4a-5b13-4ed2-a09a-b02779b87a5d admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:25.574587 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbdb4a-5b13-4ed2-a09a-b02779b87a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:25.574857 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbdb4a-5b13-4ed2-a09a-b02779b87a5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:25.576232 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fbdb4a-5b13-4ed2-a09a-b02779b87a5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:25.578214 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fbdb4a-5b13-4ed2-a09a-b02779b87a5d admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:25.634456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 818/1641] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:12:25 2022] POST /identity/v3/users => generated 308 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:25.634456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacdc81-8a99-4de3-a9bf-286ec76d30d7 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:25.639480 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-fcacdc81-8a99-4de3-a9bf-286ec76d30d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:25.642182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 824/1642] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:12:25 2022] POST /identity/v3/projects => generated 403 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:12:25.644185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbdb4a-5b13-4ed2-a09a-b02779b87a5d admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:25.772921 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:25.837416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbdb4a-5b13-4ed2-a09a-b02779b87a5d admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:25.853852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0dc33a-97ef-4224-8a89-3c9f800b2ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:25.854492 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0dc33a-97ef-4224-8a89-3c9f800b2ced admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:25.854766 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0dc33a-97ef-4224-8a89-3c9f800b2ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:25.855033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0dc33a-97ef-4224-8a89-3c9f800b2ced admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:25.855692 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0dc33a-97ef-4224-8a89-3c9f800b2ced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:25.857325 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0dc33a-97ef-4224-8a89-3c9f800b2ced admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:25.863038 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacdc81-8a99-4de3-a9bf-286ec76d30d7 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:25.872654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 825/1643] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:25.872654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0dc33a-97ef-4224-8a89-3c9f800b2ced admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:25.906927 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0dc33a-97ef-4224-8a89-3c9f800b2ced admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:25.913541 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c89bb-f33a-4257-9bee-25d75fb8dbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:25.914318 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c89bb-f33a-4257-9bee-25d75fb8dbf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:25.914520 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c89bb-f33a-4257-9bee-25d75fb8dbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:25.914791 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c89bb-f33a-4257-9bee-25d75fb8dbf9 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:25.915692 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c89bb-f33a-4257-9bee-25d75fb8dbf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:25.917954 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c89bb-f33a-4257-9bee-25d75fb8dbf9 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:25.966760 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 826/1644] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:12:25 2022] POST /identity/v3/users => generated 302 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:25.971989 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c89bb-f33a-4257-9bee-25d75fb8dbf9 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:26.000866 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c89bb-f33a-4257-9bee-25d75fb8dbf9 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:26.013067 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc916b76-f5d7-4055-8f65-42b4a41aaf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:26.015191 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc916b76-f5d7-4055-8f65-42b4a41aaf29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:26.016864 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc916b76-f5d7-4055-8f65-42b4a41aaf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:26.021650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc916b76-f5d7-4055-8f65-42b4a41aaf29 admin admin] PATH_INFO: `/v3/projects/345261108b45454eb064fff7a2fb75a9/users/caedfce7f45a4a7483cc3cb75d9f73f6/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:26.032807 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b42cc-a846-4e3b-9d45-30114b8bb2fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:26.033672 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b42cc-a846-4e3b-9d45-30114b8bb2fa service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:26.034103 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b42cc-a846-4e3b-9d45-30114b8bb2fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:26.034536 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b42cc-a846-4e3b-9d45-30114b8bb2fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:26.063494 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b42cc-a846-4e3b-9d45-30114b8bb2fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:26.065322 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b42cc-a846-4e3b-9d45-30114b8bb2fa service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:26.078291 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc916b76-f5d7-4055-8f65-42b4a41aaf29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=345261108b45454eb064fff7a2fb75a9, user_id=caedfce7f45a4a7483cc3cb75d9f73f6, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:26.083671 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 819/1645] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:26.084693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc916b76-f5d7-4055-8f65-42b4a41aaf29 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:26.086435 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b42cc-a846-4e3b-9d45-30114b8bb2fa service neutron] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:26.104167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b42cc-a846-4e3b-9d45-30114b8bb2fa service neutron] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:26.116514 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 827/1646] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:25 2022] PUT /identity/v3/projects/345261108b45454eb064fff7a2fb75a9/users/caedfce7f45a4a7483cc3cb75d9f73f6/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:26.119649 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc916b76-f5d7-4055-8f65-42b4a41aaf29 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:26.137621 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:26.254100 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f17a1-55e2-4115-be42-3a98672a9270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:26.254835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f17a1-55e2-4115-be42-3a98672a9270 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:26.255228 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f17a1-55e2-4115-be42-3a98672a9270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:26.255581 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f17a1-55e2-4115-be42-3a98672a9270 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:26.256326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f17a1-55e2-4115-be42-3a98672a9270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:26.258136 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f17a1-55e2-4115-be42-3a98672a9270 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:26.282057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 820/1647] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:26.287516 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f17a1-55e2-4115-be42-3a98672a9270 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:26.296382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc916b76-f5d7-4055-8f65-42b4a41aaf29 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:26.306942 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f17a1-55e2-4115-be42-3a98672a9270 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:26.312423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c73004-ccb1-4591-9e17-d12769ba1ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:26.316105 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c73004-ccb1-4591-9e17-d12769ba1ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:26.318182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c73004-ccb1-4591-9e17-d12769ba1ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:26.318847 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c73004-ccb1-4591-9e17-d12769ba1ac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:26.323491 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4bfcd2-488c-42ef-84c0-5b4c38e4e64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:26.324211 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4bfcd2-488c-42ef-84c0-5b4c38e4e64d admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:26.324739 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4bfcd2-488c-42ef-84c0-5b4c38e4e64d admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:26.325081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4bfcd2-488c-42ef-84c0-5b4c38e4e64d admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:26.326258 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4bfcd2-488c-42ef-84c0-5b4c38e4e64d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:26.328303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4bfcd2-488c-42ef-84c0-5b4c38e4e64d admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:26.332333 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c73004-ccb1-4591-9e17-d12769ba1ac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:26.340789 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c73004-ccb1-4591-9e17-d12769ba1ac4 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:26.360717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 821/1648] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:12:26.362210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4bfcd2-488c-42ef-84c0-5b4c38e4e64d admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:26.376244 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 828/1649] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:26.382841 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c73004-ccb1-4591-9e17-d12769ba1ac4 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:26.387764 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4bfcd2-488c-42ef-84c0-5b4c38e4e64d admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:26.394232 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a274b3-8ee9-4108-9e75-c568021212fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:26.400506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a274b3-8ee9-4108-9e75-c568021212fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:26.400863 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a274b3-8ee9-4108-9e75-c568021212fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:26.401229 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a274b3-8ee9-4108-9e75-c568021212fb admin admin] PATH_INFO: `/v3/projects/cd3b2e9eeecc479286a60a0af7596571/users/11800f62a1a64237b664cf82757506e5/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:26.421477 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c73004-ccb1-4591-9e17-d12769ba1ac4 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:26.422141 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a274b3-8ee9-4108-9e75-c568021212fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd3b2e9eeecc479286a60a0af7596571, user_id=11800f62a1a64237b664cf82757506e5, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:26.424882 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a274b3-8ee9-4108-9e75-c568021212fb admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:26.443100 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ea398-8a07-4190-b229-e1c77014bac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:26.446561 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ea398-8a07-4190-b229-e1c77014bac4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:26.446922 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ea398-8a07-4190-b229-e1c77014bac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:26.447275 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ea398-8a07-4190-b229-e1c77014bac4 admin admin] PATH_INFO: `/v3/projects/345261108b45454eb064fff7a2fb75a9/users/caedfce7f45a4a7483cc3cb75d9f73f6/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:26.449535 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ea398-8a07-4190-b229-e1c77014bac4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=345261108b45454eb064fff7a2fb75a9, user_id=caedfce7f45a4a7483cc3cb75d9f73f6, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:26.452885 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 822/1650] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:26 2022] PUT /identity/v3/projects/cd3b2e9eeecc479286a60a0af7596571/users/11800f62a1a64237b664cf82757506e5/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:26.467668 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f2654-927b-4896-96c8-0343c44ac6b3 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:26.467958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f2654-927b-4896-96c8-0343c44ac6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:26.468229 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f2654-927b-4896-96c8-0343c44ac6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:26.483407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ea398-8a07-4190-b229-e1c77014bac4 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:26.518851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-528f2654-927b-4896-96c8-0343c44ac6b3 None None] MFA Rules not processed for user `11800f62a1a64237b664cf82757506e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:26.520382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-528f2654-927b-4896-96c8-0343c44ac6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:26.536326 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 829/1651] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:26 2022] PUT /identity/v3/projects/345261108b45454eb064fff7a2fb75a9/users/caedfce7f45a4a7483cc3cb75d9f73f6/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:26.710071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 823/1652] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:26 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:26.716966 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ea398-8a07-4190-b229-e1c77014bac4 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:26.916259 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ea398-8a07-4190-b229-e1c77014bac4 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:26.925371 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6af9c0-2ab5-4875-bbef-a44d92dc8400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:26.926012 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6af9c0-2ab5-4875-bbef-a44d92dc8400 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:26.926292 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6af9c0-2ab5-4875-bbef-a44d92dc8400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:26.926656 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6af9c0-2ab5-4875-bbef-a44d92dc8400 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:26.927190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6af9c0-2ab5-4875-bbef-a44d92dc8400 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:26.929535 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6af9c0-2ab5-4875-bbef-a44d92dc8400 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:26.970908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:26.986699 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 830/1653] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:12:26 2022] POST /identity/v3/projects => generated 403 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:26.991307 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f2654-927b-4896-96c8-0343c44ac6b3 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:27.006656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f2654-927b-4896-96c8-0343c44ac6b3 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:27.214826 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cabe1b-fbb0-48db-91e1-3d24eb7aea8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:27.217407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cabe1b-fbb0-48db-91e1-3d24eb7aea8c admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:27.217407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cabe1b-fbb0-48db-91e1-3d24eb7aea8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:27.217407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cabe1b-fbb0-48db-91e1-3d24eb7aea8c admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:27.217407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cabe1b-fbb0-48db-91e1-3d24eb7aea8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:27.225674 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cabe1b-fbb0-48db-91e1-3d24eb7aea8c admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:27.287786 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:27.316413 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 824/1654] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:12:26 2022] POST /identity/v3/users => generated 308 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:27.320476 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6af9c0-2ab5-4875-bbef-a44d92dc8400 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:27.334933 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6af9c0-2ab5-4875-bbef-a44d92dc8400 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:27.341887 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389006b-e7dd-428b-b3ba-fad08ef3f500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:27.342696 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389006b-e7dd-428b-b3ba-fad08ef3f500 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:27.343248 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389006b-e7dd-428b-b3ba-fad08ef3f500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:27.343762 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389006b-e7dd-428b-b3ba-fad08ef3f500 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:27.344598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0389006b-e7dd-428b-b3ba-fad08ef3f500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:27.347283 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0389006b-e7dd-428b-b3ba-fad08ef3f500 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:27.366311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 831/1655] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:27.367642 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cabe1b-fbb0-48db-91e1-3d24eb7aea8c admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:27.382016 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cabe1b-fbb0-48db-91e1-3d24eb7aea8c admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:27.402345 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb76000-ee32-431b-abdc-2d561bf3a641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:27.403032 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb76000-ee32-431b-abdc-2d561bf3a641 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:27.403312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb76000-ee32-431b-abdc-2d561bf3a641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:27.403586 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb76000-ee32-431b-abdc-2d561bf3a641 admin admin] PATH_INFO: `/v3/projects/8c1a451664ed43969072415003a2d1d9/users/738433c143134294bd3973fc3936815e/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:27.464202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb76000-ee32-431b-abdc-2d561bf3a641 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c1a451664ed43969072415003a2d1d9, user_id=738433c143134294bd3973fc3936815e, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:27.466994 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb76000-ee32-431b-abdc-2d561bf3a641 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:27.499561 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 825/1656] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:27 2022] PUT /identity/v3/projects/8c1a451664ed43969072415003a2d1d9/users/738433c143134294bd3973fc3936815e/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:27.504518 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389006b-e7dd-428b-b3ba-fad08ef3f500 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:27.527995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389006b-e7dd-428b-b3ba-fad08ef3f500 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:27.648655 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a3ec18-aec9-4b6e-a50b-64b1660c2514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:27.649557 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3ec18-aec9-4b6e-a50b-64b1660c2514 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:27.649837 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3ec18-aec9-4b6e-a50b-64b1660c2514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:27.650159 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3ec18-aec9-4b6e-a50b-64b1660c2514 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:27.651090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a3ec18-aec9-4b6e-a50b-64b1660c2514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:27.654877 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a3ec18-aec9-4b6e-a50b-64b1660c2514 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:27.687431 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 832/1657] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:27.689369 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb76000-ee32-431b-abdc-2d561bf3a641 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:27.719785 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb76000-ee32-431b-abdc-2d561bf3a641 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:27.733181 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750433fe-1de2-4128-b84e-951f43d5436a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:27.734166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750433fe-1de2-4128-b84e-951f43d5436a admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:27.734567 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750433fe-1de2-4128-b84e-951f43d5436a admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:27.737235 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750433fe-1de2-4128-b84e-951f43d5436a admin admin] PATH_INFO: `/v3/projects/8c1a451664ed43969072415003a2d1d9/users/738433c143134294bd3973fc3936815e/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:27.741410 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750433fe-1de2-4128-b84e-951f43d5436a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c1a451664ed43969072415003a2d1d9, user_id=738433c143134294bd3973fc3936815e, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:27.746558 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750433fe-1de2-4128-b84e-951f43d5436a admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:27.783992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 826/1658] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:27 2022] PUT /identity/v3/projects/8c1a451664ed43969072415003a2d1d9/users/738433c143134294bd3973fc3936815e/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:27.787924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a6369-9eea-4431-a649-4a2bcecac333 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:27.788365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a6369-9eea-4431-a649-4a2bcecac333 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:27.788886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a6369-9eea-4431-a649-4a2bcecac333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:27.873434 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-0d2a6369-9eea-4431-a649-4a2bcecac333 None None] MFA Rules not processed for user `738433c143134294bd3973fc3936815e`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:27.876339 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-0d2a6369-9eea-4431-a649-4a2bcecac333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:28.075337 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 833/1659] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:27 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:28.093179 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750433fe-1de2-4128-b84e-951f43d5436a admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:28.095123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-750433fe-1de2-4128-b84e-951f43d5436a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:28.244279 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750433fe-1de2-4128-b84e-951f43d5436a admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:28.265744 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77dd47c-4910-499c-9aaf-75734e17fef2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:28.267061 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77dd47c-4910-499c-9aaf-75734e17fef2 service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:28.267369 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77dd47c-4910-499c-9aaf-75734e17fef2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:28.267644 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77dd47c-4910-499c-9aaf-75734e17fef2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:28.298368 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77dd47c-4910-499c-9aaf-75734e17fef2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:28.299851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77dd47c-4910-499c-9aaf-75734e17fef2 service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:28.323418 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 827/1660] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:28 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:28.356716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a6369-9eea-4431-a649-4a2bcecac333 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:28.365347 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-0d2a6369-9eea-4431-a649-4a2bcecac333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:28.534805 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a6369-9eea-4431-a649-4a2bcecac333 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:28.547504 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e656b-708b-43fb-8106-9909af7f3ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:28.548210 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e656b-708b-43fb-8106-9909af7f3ae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:28.548491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e656b-708b-43fb-8106-9909af7f3ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:28.548768 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e656b-708b-43fb-8106-9909af7f3ae3 admin admin] PATH_INFO: `/v3/projects/3a9a84610b96408189df339505ac4e2d` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:28.553076 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395e656b-708b-43fb-8106-9909af7f3ae3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a9a84610b96408189df339505ac4e2d)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:28.559695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395e656b-708b-43fb-8106-9909af7f3ae3 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:28.597305 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:28.672927 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-395e656b-708b-43fb-8106-9909af7f3ae3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a9a84610b96408189df339505ac4e2d', 'request_id': 'req-395e656b-708b-43fb-8106-9909af7f3ae3'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:28.687486 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77dd47c-4910-499c-9aaf-75734e17fef2 service cinder] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:28.690214 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-395e656b-708b-43fb-8106-9909af7f3ae3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a9a84610b96408189df339505ac4e2d', 'request_id': 'req-395e656b-708b-43fb-8106-9909af7f3ae3'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:28.702680 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-395e656b-708b-43fb-8106-9909af7f3ae3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a9a84610b96408189df339505ac4e2d', 'request_id': 'req-395e656b-708b-43fb-8106-9909af7f3ae3'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:28.705102 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77dd47c-4910-499c-9aaf-75734e17fef2 service cinder] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:28.733361 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-395e656b-708b-43fb-8106-9909af7f3ae3 admin admin] The token cache is being invalidate because project 3a9a84610b96408189df339505ac4e2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:12:28.733852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-395e656b-708b-43fb-8106-9909af7f3ae3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:28.735448 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-395e656b-708b-43fb-8106-9909af7f3ae3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:28.749127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 834/1661] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:12:28 2022] DELETE /identity/v3/projects/3a9a84610b96408189df339505ac4e2d => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:28.754862 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc5b7b-ae92-4f78-a249-9c8e649fa083 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:28.755168 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc5b7b-ae92-4f78-a249-9c8e649fa083 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:28.757005 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc5b7b-ae92-4f78-a249-9c8e649fa083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:28.813189 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-50bc5b7b-ae92-4f78-a249-9c8e649fa083 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:28.814812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-50bc5b7b-ae92-4f78-a249-9c8e649fa083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:28.895431 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb7ec47-05a7-49e1-a556-2fe185834064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:28.896449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7ec47-05a7-49e1-a556-2fe185834064 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:28.897312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7ec47-05a7-49e1-a556-2fe185834064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:28.899285 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7ec47-05a7-49e1-a556-2fe185834064 admin admin] PATH_INFO: `/v3/users/11800f62a1a64237b664cf82757506e5` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:28.901545 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb7ec47-05a7-49e1-a556-2fe185834064 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11800f62a1a64237b664cf82757506e5)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:28.909625 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb7ec47-05a7-49e1-a556-2fe185834064 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:28.975128 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 835/1662] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:28.983615 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54aa737-7e0c-4655-afae-018e86123fd5 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:28.984755 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54aa737-7e0c-4655-afae-018e86123fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:28.986177 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54aa737-7e0c-4655-afae-018e86123fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:29.016997 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cfb7ec47-05a7-49e1-a556-2fe185834064 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11800f62a1a64237b664cf82757506e5'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:29.032223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cfb7ec47-05a7-49e1-a556-2fe185834064 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11800f62a1a64237b664cf82757506e5'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:29.042009 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:29.044658 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-c54aa737-7e0c-4655-afae-018e86123fd5 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:29.048254 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-c54aa737-7e0c-4655-afae-018e86123fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:29.057357 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cfb7ec47-05a7-49e1-a556-2fe185834064 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11800f62a1a64237b664cf82757506e5'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:29.059420 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cfb7ec47-05a7-49e1-a556-2fe185834064 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11800f62a1a64237b664cf82757506e5'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:29.073038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 836/1663] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:12:29.076940 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cfb7ec47-05a7-49e1-a556-2fe185834064 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11800f62a1a64237b664cf82757506e5'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:29.077315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54aa737-7e0c-4655-afae-018e86123fd5 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:29.090840 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 828/1664] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:12:28 2022] DELETE /identity/v3/users/11800f62a1a64237b664cf82757506e5 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:29.095678 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f858ce2-358b-4259-9dd9-a97f2ae72565 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:29.095960 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f858ce2-358b-4259-9dd9-a97f2ae72565 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:29.096229 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f858ce2-358b-4259-9dd9-a97f2ae72565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:29.103130 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54aa737-7e0c-4655-afae-018e86123fd5 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:29.151589 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-4f858ce2-358b-4259-9dd9-a97f2ae72565 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:29.153093 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-4f858ce2-358b-4259-9dd9-a97f2ae72565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:29.243812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216d023-d8a4-4372-b5ec-7b614968c69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:29.244665 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216d023-d8a4-4372-b5ec-7b614968c69d admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:29.245043 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216d023-d8a4-4372-b5ec-7b614968c69d admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:29.245464 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216d023-d8a4-4372-b5ec-7b614968c69d admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:29.246242 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9216d023-d8a4-4372-b5ec-7b614968c69d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:12:29.246649 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9216d023-d8a4-4372-b5ec-7b614968c69d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:29.248392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9216d023-d8a4-4372-b5ec-7b614968c69d admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:29.266931 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 837/1665] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:12:29.272345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216d023-d8a4-4372-b5ec-7b614968c69d admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:29.285549 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216d023-d8a4-4372-b5ec-7b614968c69d admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:29.292868 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b65dcf5-c90b-4dc0-9955-a71344bfd4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:29.293742 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65dcf5-c90b-4dc0-9955-a71344bfd4b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:29.294123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65dcf5-c90b-4dc0-9955-a71344bfd4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:29.294492 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65dcf5-c90b-4dc0-9955-a71344bfd4b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:29.295263 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b65dcf5-c90b-4dc0-9955-a71344bfd4b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:29.297478 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b65dcf5-c90b-4dc0-9955-a71344bfd4b4 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:29.341016 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 829/1666] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:29.368696 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 838/1667] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:12:29 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:29.370674 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f858ce2-358b-4259-9dd9-a97f2ae72565 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:29.375798 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b65dcf5-c90b-4dc0-9955-a71344bfd4b4 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:29.378375 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-5b65dcf5-c90b-4dc0-9955-a71344bfd4b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:29.390582 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f858ce2-358b-4259-9dd9-a97f2ae72565 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:29.391237 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:29.498605 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41deec5a-ab77-4f60-abc2-95fe8ffa7981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:29.499394 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41deec5a-ab77-4f60-abc2-95fe8ffa7981 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:29.500009 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41deec5a-ab77-4f60-abc2-95fe8ffa7981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:29.500336 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41deec5a-ab77-4f60-abc2-95fe8ffa7981 admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:29.501269 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41deec5a-ab77-4f60-abc2-95fe8ffa7981 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:29.505051 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41deec5a-ab77-4f60-abc2-95fe8ffa7981 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:29.528706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b65dcf5-c90b-4dc0-9955-a71344bfd4b4 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:29.539080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f9020-826e-4c9e-aee0-5c9a5f33e5fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:29.542240 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f9020-826e-4c9e-aee0-5c9a5f33e5fe service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:29.543015 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f9020-826e-4c9e-aee0-5c9a5f33e5fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:29.543015 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f9020-826e-4c9e-aee0-5c9a5f33e5fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:29.556193 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 830/1668] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:12:29 2022] POST /identity/v3/users => generated 286 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:12:29.559683 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41deec5a-ab77-4f60-abc2-95fe8ffa7981 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:29.568770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f9020-826e-4c9e-aee0-5c9a5f33e5fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:29.572583 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f9020-826e-4c9e-aee0-5c9a5f33e5fe service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:29.580087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41deec5a-ab77-4f60-abc2-95fe8ffa7981 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:29.589431 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261714ca-e099-46fb-8b96-a19d90487cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:29.590067 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261714ca-e099-46fb-8b96-a19d90487cb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:29.591015 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261714ca-e099-46fb-8b96-a19d90487cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:29.591296 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261714ca-e099-46fb-8b96-a19d90487cb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:29.593321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261714ca-e099-46fb-8b96-a19d90487cb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:29.594994 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261714ca-e099-46fb-8b96-a19d90487cb9 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:29.598628 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 839/1669] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:29.612613 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 831/1670] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:29.616452 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f9020-826e-4c9e-aee0-5c9a5f33e5fe service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:29.629481 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f9020-826e-4c9e-aee0-5c9a5f33e5fe service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:29.643041 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd73bc-595a-4a17-9d89-7597bcba63e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:29.644806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd73bc-595a-4a17-9d89-7597bcba63e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:29.645212 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd73bc-595a-4a17-9d89-7597bcba63e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:29.645764 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd73bc-595a-4a17-9d89-7597bcba63e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:29.647006 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd73bc-595a-4a17-9d89-7597bcba63e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:29.651335 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd73bc-595a-4a17-9d89-7597bcba63e7 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:29.680157 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 840/1671] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:29.680846 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261714ca-e099-46fb-8b96-a19d90487cb9 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:29.709286 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261714ca-e099-46fb-8b96-a19d90487cb9 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:29.728380 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4f750-f430-496c-a146-ea9ef5f88982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:29.729651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4f750-f430-496c-a146-ea9ef5f88982 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:29.733040 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4f750-f430-496c-a146-ea9ef5f88982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:29.733398 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4f750-f430-496c-a146-ea9ef5f88982 admin admin] PATH_INFO: `/v3/projects/8eb45cf23162412fa5e10a1b30c03a2d/users/acabfd6344254c38ae642ba462ab4cab/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:29.783403 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4f750-f430-496c-a146-ea9ef5f88982 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eb45cf23162412fa5e10a1b30c03a2d, user_id=acabfd6344254c38ae642ba462ab4cab, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:29.789822 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4f750-f430-496c-a146-ea9ef5f88982 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:29.823619 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 832/1672] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:29 2022] PUT /identity/v3/projects/8eb45cf23162412fa5e10a1b30c03a2d/users/acabfd6344254c38ae642ba462ab4cab/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:29.827754 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d00a23-25b9-47d3-8626-f2ad164469ad None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:29.828312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d00a23-25b9-47d3-8626-f2ad164469ad None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:29.828971 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d00a23-25b9-47d3-8626-f2ad164469ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:29.896266 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-c5d00a23-25b9-47d3-8626-f2ad164469ad None None] MFA Rules not processed for user `acabfd6344254c38ae642ba462ab4cab`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:29.897991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-c5d00a23-25b9-47d3-8626-f2ad164469ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:30.079540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 841/1673] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:30.083878 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4f750-f430-496c-a146-ea9ef5f88982 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:30.203505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:30.321627 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4f750-f430-496c-a146-ea9ef5f88982 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:30.355928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23fcaf9-0f4a-4549-a85b-6f0f7cfd52ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:30.358814 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fcaf9-0f4a-4549-a85b-6f0f7cfd52ac admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:30.359199 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fcaf9-0f4a-4549-a85b-6f0f7cfd52ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:30.361065 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fcaf9-0f4a-4549-a85b-6f0f7cfd52ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:30.362532 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23fcaf9-0f4a-4549-a85b-6f0f7cfd52ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:30.370167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23fcaf9-0f4a-4549-a85b-6f0f7cfd52ac admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:30.458277 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 833/1674] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:12:30 2022] POST /identity/v3/projects => generated 371 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:30.460487 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d00a23-25b9-47d3-8626-f2ad164469ad None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:30.482999 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d00a23-25b9-47d3-8626-f2ad164469ad None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:30.658326 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028afcf-b037-4e2e-bd7e-24b3c01ca57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:30.658895 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028afcf-b037-4e2e-bd7e-24b3c01ca57b admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:30.662938 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028afcf-b037-4e2e-bd7e-24b3c01ca57b admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:30.663227 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028afcf-b037-4e2e-bd7e-24b3c01ca57b admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:30.663967 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c028afcf-b037-4e2e-bd7e-24b3c01ca57b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:30.665977 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c028afcf-b037-4e2e-bd7e-24b3c01ca57b admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:30.714183 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 842/1675] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:12:30 2022] POST /identity/v3/users => generated 292 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:30.715807 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23fcaf9-0f4a-4549-a85b-6f0f7cfd52ac admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:30.729968 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23fcaf9-0f4a-4549-a85b-6f0f7cfd52ac admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:30.744823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a83ca-53ee-4211-8d22-afc36ae4ef22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:30.745482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a83ca-53ee-4211-8d22-afc36ae4ef22 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:30.745766 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a83ca-53ee-4211-8d22-afc36ae4ef22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:30.746097 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a83ca-53ee-4211-8d22-afc36ae4ef22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:30.746688 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8a83ca-53ee-4211-8d22-afc36ae4ef22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:30.748794 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8a83ca-53ee-4211-8d22-afc36ae4ef22 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:30.777751 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 834/1676] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:30.782183 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028afcf-b037-4e2e-bd7e-24b3c01ca57b admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:30.798087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028afcf-b037-4e2e-bd7e-24b3c01ca57b admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:30.811444 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62f3c6-e485-4fff-b9ec-221c22f40e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:30.812173 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62f3c6-e485-4fff-b9ec-221c22f40e80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:30.812438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62f3c6-e485-4fff-b9ec-221c22f40e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:30.812778 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62f3c6-e485-4fff-b9ec-221c22f40e80 admin admin] PATH_INFO: `/v3/projects/939bc03fc3a14c998b288d6a07c0ec2d/users/e149dd8d7ae742528472de631131c072/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:30.849508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:30.858533 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed62f3c6-e485-4fff-b9ec-221c22f40e80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=939bc03fc3a14c998b288d6a07c0ec2d, user_id=e149dd8d7ae742528472de631131c072, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:30.861020 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed62f3c6-e485-4fff-b9ec-221c22f40e80 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:30.895835 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 843/1677] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:30 2022] PUT /identity/v3/projects/939bc03fc3a14c998b288d6a07c0ec2d/users/e149dd8d7ae742528472de631131c072/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:30.900000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a83ca-53ee-4211-8d22-afc36ae4ef22 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:30.903384 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7595b-4c8d-4608-b543-6073fb8b225a None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:30.903671 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7595b-4c8d-4608-b543-6073fb8b225a None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:30.904616 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7595b-4c8d-4608-b543-6073fb8b225a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:30.915642 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a83ca-53ee-4211-8d22-afc36ae4ef22 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:30.979304 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-61b7595b-4c8d-4608-b543-6073fb8b225a None None] MFA Rules not processed for user `caedfce7f45a4a7483cc3cb75d9f73f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:30.984619 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-61b7595b-4c8d-4608-b543-6073fb8b225a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:31.034121 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68172b9b-918b-4326-9b0b-b0c0dae8a768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:31.037817 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68172b9b-918b-4326-9b0b-b0c0dae8a768 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:31.039735 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68172b9b-918b-4326-9b0b-b0c0dae8a768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:31.040259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68172b9b-918b-4326-9b0b-b0c0dae8a768 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:31.041128 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68172b9b-918b-4326-9b0b-b0c0dae8a768 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:31.052444 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68172b9b-918b-4326-9b0b-b0c0dae8a768 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:31.068102 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 835/1678] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:12:31.074633 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68172b9b-918b-4326-9b0b-b0c0dae8a768 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:31.106127 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68172b9b-918b-4326-9b0b-b0c0dae8a768 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:31.125685 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca89630-af02-4d44-9780-b27157279232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:31.126395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca89630-af02-4d44-9780-b27157279232 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:31.126672 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca89630-af02-4d44-9780-b27157279232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:31.126942 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca89630-af02-4d44-9780-b27157279232 admin admin] PATH_INFO: `/v3/projects/939bc03fc3a14c998b288d6a07c0ec2d/users/e149dd8d7ae742528472de631131c072/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:31.129669 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca89630-af02-4d44-9780-b27157279232 admin admin] RBAC: Authorizing `identity:create_grant(project_id=939bc03fc3a14c998b288d6a07c0ec2d, user_id=e149dd8d7ae742528472de631131c072, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:31.135092 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca89630-af02-4d44-9780-b27157279232 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:31.156039 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 844/1679] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:30 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:31.166901 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 836/1680] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:31 2022] PUT /identity/v3/projects/939bc03fc3a14c998b288d6a07c0ec2d/users/e149dd8d7ae742528472de631131c072/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:31.177663 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9177d6-89db-432b-affe-ac6df7eb40ab None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:31.177944 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9177d6-89db-432b-affe-ac6df7eb40ab None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:31.178217 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9177d6-89db-432b-affe-ac6df7eb40ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:31.231428 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-bb9177d6-89db-432b-affe-ac6df7eb40ab None None] MFA Rules not processed for user `e149dd8d7ae742528472de631131c072`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:31.234465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-bb9177d6-89db-432b-affe-ac6df7eb40ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:31.295108 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca89630-af02-4d44-9780-b27157279232 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:31.297083 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-dca89630-af02-4d44-9780-b27157279232 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:31.444354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 845/1681] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:31 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:31.453878 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9177d6-89db-432b-affe-ac6df7eb40ab None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:31.456092 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-bb9177d6-89db-432b-affe-ac6df7eb40ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:31.484666 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca89630-af02-4d44-9780-b27157279232 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:31.500393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70098c82-3b5a-4d33-84e0-964c9fa6c2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:31.501113 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70098c82-3b5a-4d33-84e0-964c9fa6c2c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:31.501419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70098c82-3b5a-4d33-84e0-964c9fa6c2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:31.501689 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70098c82-3b5a-4d33-84e0-964c9fa6c2c6 admin admin] PATH_INFO: `/v3/users/28c98119f52c4d2ab43ffb539483a823` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:31.502996 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70098c82-3b5a-4d33-84e0-964c9fa6c2c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28c98119f52c4d2ab43ffb539483a823)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:31.508334 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70098c82-3b5a-4d33-84e0-964c9fa6c2c6 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:31.583986 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:31.609344 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-70098c82-3b5a-4d33-84e0-964c9fa6c2c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28c98119f52c4d2ab43ffb539483a823'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:31.621526 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9177d6-89db-432b-affe-ac6df7eb40ab None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:31.631237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-70098c82-3b5a-4d33-84e0-964c9fa6c2c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28c98119f52c4d2ab43ffb539483a823'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:31.633835 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad245f63-6125-4cc7-bf58-fb5a3920bdb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:31.635616 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad245f63-6125-4cc7-bf58-fb5a3920bdb3 service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:31.635898 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad245f63-6125-4cc7-bf58-fb5a3920bdb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:31.636169 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad245f63-6125-4cc7-bf58-fb5a3920bdb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:31.656914 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-70098c82-3b5a-4d33-84e0-964c9fa6c2c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28c98119f52c4d2ab43ffb539483a823'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:31.658939 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-70098c82-3b5a-4d33-84e0-964c9fa6c2c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28c98119f52c4d2ab43ffb539483a823'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:31.660752 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad245f63-6125-4cc7-bf58-fb5a3920bdb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:31.663374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad245f63-6125-4cc7-bf58-fb5a3920bdb3 service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:31.680249 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-70098c82-3b5a-4d33-84e0-964c9fa6c2c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28c98119f52c4d2ab43ffb539483a823'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:31.687584 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 846/1682] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:31 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:31.691805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 837/1683] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:12:31 2022] DELETE /identity/v3/users/28c98119f52c4d2ab43ffb539483a823 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:31.697590 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab721be-4dbe-40c9-9554-5a4563aef5b4 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:31.697791 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab721be-4dbe-40c9-9554-5a4563aef5b4 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:31.698087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab721be-4dbe-40c9-9554-5a4563aef5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:31.715067 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:31.753970 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-8ab721be-4dbe-40c9-9554-5a4563aef5b4 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:31.755261 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-8ab721be-4dbe-40c9-9554-5a4563aef5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:31.973194 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 847/1684] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:32.200955 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70098c82-3b5a-4d33-84e0-964c9fa6c2c6 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:32.204141 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-70098c82-3b5a-4d33-84e0-964c9fa6c2c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:32.443640 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70098c82-3b5a-4d33-84e0-964c9fa6c2c6 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:32.462502 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829bf3a3-790f-4641-bd07-976b297eee96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:32.463319 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bf3a3-790f-4641-bd07-976b297eee96 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:32.471566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bf3a3-790f-4641-bd07-976b297eee96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:32.473227 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bf3a3-790f-4641-bd07-976b297eee96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:32.495728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829bf3a3-790f-4641-bd07-976b297eee96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:32.497558 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829bf3a3-790f-4641-bd07-976b297eee96 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:32.530682 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 838/1685] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:32.677128 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab721be-4dbe-40c9-9554-5a4563aef5b4 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:32.680691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-8ab721be-4dbe-40c9-9554-5a4563aef5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:32.705193 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab721be-4dbe-40c9-9554-5a4563aef5b4 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:32.712569 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c27d52-34fd-40af-a69a-79bbf050a668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:32.713691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c27d52-34fd-40af-a69a-79bbf050a668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:32.714089 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c27d52-34fd-40af-a69a-79bbf050a668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:32.714439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c27d52-34fd-40af-a69a-79bbf050a668 admin admin] PATH_INFO: `/v3/users/acabfd6344254c38ae642ba462ab4cab` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:32.715721 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c27d52-34fd-40af-a69a-79bbf050a668 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acabfd6344254c38ae642ba462ab4cab)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:32.718632 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c27d52-34fd-40af-a69a-79bbf050a668 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:32.801849 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e6c27d52-34fd-40af-a69a-79bbf050a668 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acabfd6344254c38ae642ba462ab4cab'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:32.815477 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e6c27d52-34fd-40af-a69a-79bbf050a668 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acabfd6344254c38ae642ba462ab4cab'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:32.834970 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e6c27d52-34fd-40af-a69a-79bbf050a668 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acabfd6344254c38ae642ba462ab4cab'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:32.835920 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e6c27d52-34fd-40af-a69a-79bbf050a668 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acabfd6344254c38ae642ba462ab4cab'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:32.847405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e6c27d52-34fd-40af-a69a-79bbf050a668 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acabfd6344254c38ae642ba462ab4cab'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:32.860952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba89588-4c77-48ec-9065-f03275cf236c None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:32.861255 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba89588-4c77-48ec-9065-f03275cf236c None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:32.861525 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba89588-4c77-48ec-9065-f03275cf236c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:32.868278 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 848/1686] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:12:32 2022] DELETE /identity/v3/users/acabfd6344254c38ae642ba462ab4cab => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:32.889500 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-4ba89588-4c77-48ec-9065-f03275cf236c None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:32.890774 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-4ba89588-4c77-48ec-9065-f03275cf236c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:33.011228 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 839/1687] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:33.057804 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c27d52-34fd-40af-a69a-79bbf050a668 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:33.061654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-e6c27d52-34fd-40af-a69a-79bbf050a668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:33.254198 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c27d52-34fd-40af-a69a-79bbf050a668 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:33.266419 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82443613-a7af-4d7d-9c2b-eeb14ec44b80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:33.267150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82443613-a7af-4d7d-9c2b-eeb14ec44b80 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:33.267490 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82443613-a7af-4d7d-9c2b-eeb14ec44b80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:33.267845 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82443613-a7af-4d7d-9c2b-eeb14ec44b80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:33.282311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82443613-a7af-4d7d-9c2b-eeb14ec44b80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:33.284462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82443613-a7af-4d7d-9c2b-eeb14ec44b80 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:33.297577 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 849/1688] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:33.395430 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba89588-4c77-48ec-9065-f03275cf236c None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:33.397527 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-4ba89588-4c77-48ec-9065-f03275cf236c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:33.422489 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba89588-4c77-48ec-9065-f03275cf236c None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:33.441583 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e3526-51ae-456b-b6f4-e0011d1a67b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:33.442254 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e3526-51ae-456b-b6f4-e0011d1a67b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:33.442624 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e3526-51ae-456b-b6f4-e0011d1a67b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:33.442899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e3526-51ae-456b-b6f4-e0011d1a67b8 admin admin] PATH_INFO: `/v3/projects/cd3b2e9eeecc479286a60a0af7596571` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:33.444188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e3526-51ae-456b-b6f4-e0011d1a67b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd3b2e9eeecc479286a60a0af7596571)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:33.447059 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e3526-51ae-456b-b6f4-e0011d1a67b8 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:33.544010 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:33.546227 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-4a7e3526-51ae-456b-b6f4-e0011d1a67b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd3b2e9eeecc479286a60a0af7596571', 'request_id': 'req-4a7e3526-51ae-456b-b6f4-e0011d1a67b8'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:33.563249 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-4a7e3526-51ae-456b-b6f4-e0011d1a67b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd3b2e9eeecc479286a60a0af7596571', 'request_id': 'req-4a7e3526-51ae-456b-b6f4-e0011d1a67b8'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:33.571110 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-4a7e3526-51ae-456b-b6f4-e0011d1a67b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd3b2e9eeecc479286a60a0af7596571', 'request_id': 'req-4a7e3526-51ae-456b-b6f4-e0011d1a67b8'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:33.598013 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-4a7e3526-51ae-456b-b6f4-e0011d1a67b8 admin admin] The token cache is being invalidate because project cd3b2e9eeecc479286a60a0af7596571 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:12:33.599193 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-4a7e3526-51ae-456b-b6f4-e0011d1a67b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:33.600320 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-4a7e3526-51ae-456b-b6f4-e0011d1a67b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:33.605030 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 840/1689] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:12:33 2022] DELETE /identity/v3/projects/cd3b2e9eeecc479286a60a0af7596571 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:33.610123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82443613-a7af-4d7d-9c2b-eeb14ec44b80 service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:33.623208 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82443613-a7af-4d7d-9c2b-eeb14ec44b80 service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:33.657631 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:33.749931 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e63a0-4c33-4df3-84bb-88021de32768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:33.751416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e63a0-4c33-4df3-84bb-88021de32768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:33.753033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e63a0-4c33-4df3-84bb-88021de32768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:33.753522 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e63a0-4c33-4df3-84bb-88021de32768 admin admin] PATH_INFO: `/v3/users/738433c143134294bd3973fc3936815e` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:33.755359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e63a0-4c33-4df3-84bb-88021de32768 admin admin] RBAC: Authorizing `identity:delete_user(user_id=738433c143134294bd3973fc3936815e)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:33.758560 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e63a0-4c33-4df3-84bb-88021de32768 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:33.865062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-539e63a0-4c33-4df3-84bb-88021de32768 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '738433c143134294bd3973fc3936815e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:33.890609 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-539e63a0-4c33-4df3-84bb-88021de32768 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '738433c143134294bd3973fc3936815e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:33.909207 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-539e63a0-4c33-4df3-84bb-88021de32768 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '738433c143134294bd3973fc3936815e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:33.910987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-539e63a0-4c33-4df3-84bb-88021de32768 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '738433c143134294bd3973fc3936815e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:33.927234 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-539e63a0-4c33-4df3-84bb-88021de32768 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '738433c143134294bd3973fc3936815e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:33.943033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 850/1690] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:12:33 2022] DELETE /identity/v3/users/738433c143134294bd3973fc3936815e => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:34.479027 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e253d-67b9-4813-8763-e060b736cd25 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:34.479582 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e253d-67b9-4813-8763-e060b736cd25 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:34.479582 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e253d-67b9-4813-8763-e060b736cd25 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:34.481236 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 841/1691] 10.222.0.44 () {54 vars in 921 bytes} [Sun Sep 11 15:12:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:12:34.511016 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18e187-2919-4178-91c7-ebc117f2673c None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:34.512101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18e187-2919-4178-91c7-ebc117f2673c None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:34.513804 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18e187-2919-4178-91c7-ebc117f2673c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:34.544401 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-cd18e187-2919-4178-91c7-ebc117f2673c None None] MFA Rules not processed for user `ead06362f8a2444bbfe1405cd602f1c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:34.547247 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-cd18e187-2919-4178-91c7-ebc117f2673c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:34.566183 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e253d-67b9-4813-8763-e060b736cd25 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:34.570729 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-674e253d-67b9-4813-8763-e060b736cd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:34.810924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 851/1692] 10.222.0.44 () {58 vars in 1018 bytes} [Sun Sep 11 15:12:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:34.817052 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff4771-4548-427f-b957-262a9d44773d None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:34.817435 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff4771-4548-427f-b957-262a9d44773d None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:34.820820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff4771-4548-427f-b957-262a9d44773d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:34.824521 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e253d-67b9-4813-8763-e060b736cd25 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:34.832827 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215dfd0-7ce1-4b28-ae3e-7e89291c0866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:34.834908 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215dfd0-7ce1-4b28-ae3e-7e89291c0866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:34.835618 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215dfd0-7ce1-4b28-ae3e-7e89291c0866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:34.836435 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215dfd0-7ce1-4b28-ae3e-7e89291c0866 admin admin] PATH_INFO: `/v3/projects/7860f1640b1d4d9ebbdff61db2c56d31` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:34.838437 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d215dfd0-7ce1-4b28-ae3e-7e89291c0866 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7860f1640b1d4d9ebbdff61db2c56d31)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:34.840699 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d215dfd0-7ce1-4b28-ae3e-7e89291c0866 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:34.912371 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-c0ff4771-4548-427f-b957-262a9d44773d None None] MFA Rules not processed for user `3eb721f100f24930bafd056d29048561`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:34.915804 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-c0ff4771-4548-427f-b957-262a9d44773d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:35.018360 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d215dfd0-7ce1-4b28-ae3e-7e89291c0866 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7860f1640b1d4d9ebbdff61db2c56d31', 'request_id': 'req-d215dfd0-7ce1-4b28-ae3e-7e89291c0866'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:35.030245 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:35.054695 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d215dfd0-7ce1-4b28-ae3e-7e89291c0866 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7860f1640b1d4d9ebbdff61db2c56d31', 'request_id': 'req-d215dfd0-7ce1-4b28-ae3e-7e89291c0866'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:35.072903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d215dfd0-7ce1-4b28-ae3e-7e89291c0866 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7860f1640b1d4d9ebbdff61db2c56d31', 'request_id': 'req-d215dfd0-7ce1-4b28-ae3e-7e89291c0866'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:35.117814 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d215dfd0-7ce1-4b28-ae3e-7e89291c0866 admin admin] The token cache is being invalidate because project 7860f1640b1d4d9ebbdff61db2c56d31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:12:35.121099 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d215dfd0-7ce1-4b28-ae3e-7e89291c0866 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:35.124364 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d215dfd0-7ce1-4b28-ae3e-7e89291c0866 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:35.132738 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 842/1693] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:12:34 2022] DELETE /identity/v3/projects/7860f1640b1d4d9ebbdff61db2c56d31 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:35.135679 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215dfd0-7ce1-4b28-ae3e-7e89291c0866 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:35.139648 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-d215dfd0-7ce1-4b28-ae3e-7e89291c0866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:35.186905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 852/1694] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:35.188869 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff4771-4548-427f-b957-262a9d44773d None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:35.191829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-c0ff4771-4548-427f-b957-262a9d44773d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:35.229538 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff4771-4548-427f-b957-262a9d44773d None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:35.256040 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215dfd0-7ce1-4b28-ae3e-7e89291c0866 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:35.279763 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c5f4e-c694-4212-a24f-03410b27262a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:35.280646 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c5f4e-c694-4212-a24f-03410b27262a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:35.280794 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c5f4e-c694-4212-a24f-03410b27262a admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:35.280957 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c5f4e-c694-4212-a24f-03410b27262a admin admin] PATH_INFO: `/v3/projects/8eb45cf23162412fa5e10a1b30c03a2d` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:35.294199 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49c5f4e-c694-4212-a24f-03410b27262a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eb45cf23162412fa5e10a1b30c03a2d)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:35.296391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49c5f4e-c694-4212-a24f-03410b27262a admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:35.308558 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:35.391975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-e49c5f4e-c694-4212-a24f-03410b27262a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eb45cf23162412fa5e10a1b30c03a2d', 'request_id': 'req-e49c5f4e-c694-4212-a24f-03410b27262a'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:35.413089 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-e49c5f4e-c694-4212-a24f-03410b27262a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eb45cf23162412fa5e10a1b30c03a2d', 'request_id': 'req-e49c5f4e-c694-4212-a24f-03410b27262a'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:35.421822 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-e49c5f4e-c694-4212-a24f-03410b27262a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eb45cf23162412fa5e10a1b30c03a2d', 'request_id': 'req-e49c5f4e-c694-4212-a24f-03410b27262a'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:35.424528 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d943e4-1c7e-490e-8fb4-f780988e8289 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead06362f8a2444bbfe1405cd602f1c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:35.425680 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d943e4-1c7e-490e-8fb4-f780988e8289 service placement] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:35.426260 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d943e4-1c7e-490e-8fb4-f780988e8289 service placement] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:35.426713 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d943e4-1c7e-490e-8fb4-f780988e8289 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:35.435501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-69d943e4-1c7e-490e-8fb4-f780988e8289 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:35.451076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-e49c5f4e-c694-4212-a24f-03410b27262a admin admin] The token cache is being invalidate because project 8eb45cf23162412fa5e10a1b30c03a2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:12:35.451374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-e49c5f4e-c694-4212-a24f-03410b27262a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:35.452116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-e49c5f4e-c694-4212-a24f-03410b27262a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:35.457832 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 843/1695] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:12:35 2022] DELETE /identity/v3/projects/8eb45cf23162412fa5e10a1b30c03a2d => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:35.457832 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0235e10-136f-48c5-a7d4-1760ee2583db None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:35.457832 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0235e10-136f-48c5-a7d4-1760ee2583db None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:35.457832 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0235e10-136f-48c5-a7d4-1760ee2583db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:35.466661 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d943e4-1c7e-490e-8fb4-f780988e8289 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:35.468963 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d943e4-1c7e-490e-8fb4-f780988e8289 service placement] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:35.505037 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 853/1696] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:35.506293 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-c0235e10-136f-48c5-a7d4-1760ee2583db None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:35.507607 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c0235e10-136f-48c5-a7d4-1760ee2583db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:35.509943 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdd0e0-8170-4af2-8eb3-9e5f5b77a941 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:35.510818 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdd0e0-8170-4af2-8eb3-9e5f5b77a941 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:35.511504 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdd0e0-8170-4af2-8eb3-9e5f5b77a941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:35.566203 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-25bdd0e0-8170-4af2-8eb3-9e5f5b77a941 None None] MFA Rules not processed for user `3eb721f100f24930bafd056d29048561`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:35.567805 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-25bdd0e0-8170-4af2-8eb3-9e5f5b77a941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:35.661493 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 844/1697] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:35.668614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0235e10-136f-48c5-a7d4-1760ee2583db None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:35.669212 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c0235e10-136f-48c5-a7d4-1760ee2583db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:35.694077 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0235e10-136f-48c5-a7d4-1760ee2583db None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:35.714703 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83171e46-3bbb-46b9-9ace-2467e8c0d8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:35.715377 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83171e46-3bbb-46b9-9ace-2467e8c0d8a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:35.715773 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83171e46-3bbb-46b9-9ace-2467e8c0d8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:35.716050 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83171e46-3bbb-46b9-9ace-2467e8c0d8a8 admin admin] PATH_INFO: `/v3/users/e149dd8d7ae742528472de631131c072` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:35.717529 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83171e46-3bbb-46b9-9ace-2467e8c0d8a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e149dd8d7ae742528472de631131c072)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:35.719500 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83171e46-3bbb-46b9-9ace-2467e8c0d8a8 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:35.776974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 854/1698] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:35.776974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90f13f-03dd-4a1a-828c-ac39a3802efc None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:35.776974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90f13f-03dd-4a1a-828c-ac39a3802efc None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:35.776974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90f13f-03dd-4a1a-828c-ac39a3802efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:35.833254 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-ab90f13f-03dd-4a1a-828c-ac39a3802efc None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:35.834551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-ab90f13f-03dd-4a1a-828c-ac39a3802efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:35.858961 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-83171e46-3bbb-46b9-9ace-2467e8c0d8a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e149dd8d7ae742528472de631131c072'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:35.869362 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 855/1699] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:35.870631 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab90f13f-03dd-4a1a-828c-ac39a3802efc None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:35.872239 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-ab90f13f-03dd-4a1a-828c-ac39a3802efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:35.881582 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-83171e46-3bbb-46b9-9ace-2467e8c0d8a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e149dd8d7ae742528472de631131c072'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:35.921836 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-83171e46-3bbb-46b9-9ace-2467e8c0d8a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e149dd8d7ae742528472de631131c072'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:35.927753 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-83171e46-3bbb-46b9-9ace-2467e8c0d8a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e149dd8d7ae742528472de631131c072'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:35.945124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-83171e46-3bbb-46b9-9ace-2467e8c0d8a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e149dd8d7ae742528472de631131c072'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:35.960287 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 845/1700] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:12:35 2022] DELETE /identity/v3/users/e149dd8d7ae742528472de631131c072 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:35.961443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83171e46-3bbb-46b9-9ace-2467e8c0d8a8 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:35.964800 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-83171e46-3bbb-46b9-9ace-2467e8c0d8a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:35.994816 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:36.003056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:36.076848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab90f13f-03dd-4a1a-828c-ac39a3802efc None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:36.139422 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83171e46-3bbb-46b9-9ace-2467e8c0d8a8 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:36.152223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388b361-a822-445d-9370-b88a18d9a98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:36.152949 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388b361-a822-445d-9370-b88a18d9a98d admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:36.153339 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388b361-a822-445d-9370-b88a18d9a98d admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:36.154099 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388b361-a822-445d-9370-b88a18d9a98d admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:36.154409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388b361-a822-445d-9370-b88a18d9a98d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:12:36.155041 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388b361-a822-445d-9370-b88a18d9a98d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:36.156690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388b361-a822-445d-9370-b88a18d9a98d admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:36.184102 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 846/1701] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:12:36.184102 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388b361-a822-445d-9370-b88a18d9a98d admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:36.194149 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-3388b361-a822-445d-9370-b88a18d9a98d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:36.215224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388b361-a822-445d-9370-b88a18d9a98d admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:36.219505 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff444f1-4f5e-4d30-889b-8131270cbfa8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:36.220263 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff444f1-4f5e-4d30-889b-8131270cbfa8 service nova] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:36.220471 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff444f1-4f5e-4d30-889b-8131270cbfa8 service nova] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:36.220739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff444f1-4f5e-4d30-889b-8131270cbfa8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:36.226774 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be070816-ee1f-4ed9-a1b7-2988346a46ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:36.227627 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be070816-ee1f-4ed9-a1b7-2988346a46ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:36.228007 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be070816-ee1f-4ed9-a1b7-2988346a46ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:36.228392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be070816-ee1f-4ed9-a1b7-2988346a46ca admin admin] PATH_INFO: `/v3/projects/8c1a451664ed43969072415003a2d1d9` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:36.230000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be070816-ee1f-4ed9-a1b7-2988346a46ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c1a451664ed43969072415003a2d1d9)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:36.232920 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-dff444f1-4f5e-4d30-889b-8131270cbfa8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:36.235280 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be070816-ee1f-4ed9-a1b7-2988346a46ca admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:36.364851 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff444f1-4f5e-4d30-889b-8131270cbfa8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:36.365280 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-be070816-ee1f-4ed9-a1b7-2988346a46ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c1a451664ed43969072415003a2d1d9', 'request_id': 'req-be070816-ee1f-4ed9-a1b7-2988346a46ca'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:36.367725 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff444f1-4f5e-4d30-889b-8131270cbfa8 service nova] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:36.380181 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 856/1702] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:36.382960 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c974255-fb70-461d-8924-ecbd2146f0be None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:36.383495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c974255-fb70-461d-8924-ecbd2146f0be None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:36.383898 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c974255-fb70-461d-8924-ecbd2146f0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:36.389286 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-be070816-ee1f-4ed9-a1b7-2988346a46ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c1a451664ed43969072415003a2d1d9', 'request_id': 'req-be070816-ee1f-4ed9-a1b7-2988346a46ca'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:36.410212 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-be070816-ee1f-4ed9-a1b7-2988346a46ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c1a451664ed43969072415003a2d1d9', 'request_id': 'req-be070816-ee1f-4ed9-a1b7-2988346a46ca'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:36.436785 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-8c974255-fb70-461d-8924-ecbd2146f0be None None] MFA Rules not processed for user `5e4f8b13b66449cf849ff9cb73938c4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:36.440157 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-be070816-ee1f-4ed9-a1b7-2988346a46ca admin admin] The token cache is being invalidate because project 8c1a451664ed43969072415003a2d1d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:12:36.440571 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-be070816-ee1f-4ed9-a1b7-2988346a46ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:36.444568 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-be070816-ee1f-4ed9-a1b7-2988346a46ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:36.447049 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 847/1703] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:12:36 2022] DELETE /identity/v3/projects/8c1a451664ed43969072415003a2d1d9 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:36.455343 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeebc70-d032-4786-bd4d-d5e6159a474d None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:36.460098 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeebc70-d032-4786-bd4d-d5e6159a474d None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:36.460098 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeebc70-d032-4786-bd4d-d5e6159a474d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:36.460660 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-8c974255-fb70-461d-8924-ecbd2146f0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:36.560485 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-ceeebc70-d032-4786-bd4d-d5e6159a474d None None] MFA Rules not processed for user `c44105f2d90e469fb03f311a4cf1476f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:36.562568 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-ceeebc70-d032-4786-bd4d-d5e6159a474d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:36.585311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 857/1704] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:36.588909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c974255-fb70-461d-8924-ecbd2146f0be None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:36.591405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-8c974255-fb70-461d-8924-ecbd2146f0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:36.626913 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:36.687579 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c974255-fb70-461d-8924-ecbd2146f0be None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:36.696377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb6595-7875-4cd7-a5a8-ed7dc0caffee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:36.697448 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb6595-7875-4cd7-a5a8-ed7dc0caffee admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:36.697916 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb6595-7875-4cd7-a5a8-ed7dc0caffee admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:36.698375 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb6595-7875-4cd7-a5a8-ed7dc0caffee admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:36.699524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb6595-7875-4cd7-a5a8-ed7dc0caffee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:36.722047 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb6595-7875-4cd7-a5a8-ed7dc0caffee admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:36.749682 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 848/1705] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:36.751629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290de86-f6c5-42dc-b5cb-5402388f9713 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:36.752129 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290de86-f6c5-42dc-b5cb-5402388f9713 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:36.752681 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290de86-f6c5-42dc-b5cb-5402388f9713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:36.787173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-7290de86-f6c5-42dc-b5cb-5402388f9713 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:36.788593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-7290de86-f6c5-42dc-b5cb-5402388f9713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:36.801095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 858/1706] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:12:36 2022] POST /identity/v3/projects => generated 363 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:36.805085 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb6595-7875-4cd7-a5a8-ed7dc0caffee admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:36.806976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-e7cb6595-7875-4cd7-a5a8-ed7dc0caffee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:36.821207 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 849/1707] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:36.822311 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7290de86-f6c5-42dc-b5cb-5402388f9713 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:36.837311 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7290de86-f6c5-42dc-b5cb-5402388f9713 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:36.847105 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:36.946058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bdb693-1351-4419-92d7-57879bccfd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:36.947354 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bdb693-1351-4419-92d7-57879bccfd2f admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:36.947754 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bdb693-1351-4419-92d7-57879bccfd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:36.948505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bdb693-1351-4419-92d7-57879bccfd2f admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:36.949365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bdb693-1351-4419-92d7-57879bccfd2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:36.951946 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bdb693-1351-4419-92d7-57879bccfd2f admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:36.979310 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb6595-7875-4cd7-a5a8-ed7dc0caffee admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:36.995628 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e557c6-a607-444b-8365-2da9f77d4ff3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:36.996125 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e557c6-a607-444b-8365-2da9f77d4ff3 service nova] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:36.996396 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e557c6-a607-444b-8365-2da9f77d4ff3 service nova] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:36.996658 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e557c6-a607-444b-8365-2da9f77d4ff3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:37.008773 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 850/1708] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:12:36 2022] POST /identity/v3/users => generated 282 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:37.011997 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa111e-2f92-4abb-a22b-ee8314bff7dc None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:37.012821 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa111e-2f92-4abb-a22b-ee8314bff7dc None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:37.013415 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa111e-2f92-4abb-a22b-ee8314bff7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:37.088974 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-8dfa111e-2f92-4abb-a22b-ee8314bff7dc None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:37.121219 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e557c6-a607-444b-8365-2da9f77d4ff3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:37.122902 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e557c6-a607-444b-8365-2da9f77d4ff3 service nova] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:37.126856 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-8dfa111e-2f92-4abb-a22b-ee8314bff7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:37.148953 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 859/1709] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:36 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:37.149904 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e557c6-a607-444b-8365-2da9f77d4ff3 service nova] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:37.178023 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e557c6-a607-444b-8365-2da9f77d4ff3 service nova] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:37.191859 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867b4da-9d4b-4931-92b7-bb716695f1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:37.192508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867b4da-9d4b-4931-92b7-bb716695f1a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:37.192822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867b4da-9d4b-4931-92b7-bb716695f1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:37.193764 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867b4da-9d4b-4931-92b7-bb716695f1a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:37.194218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7867b4da-9d4b-4931-92b7-bb716695f1a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:37.196186 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 851/1710] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:37.196654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa111e-2f92-4abb-a22b-ee8314bff7dc None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:37.198983 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7867b4da-9d4b-4931-92b7-bb716695f1a7 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:37.205169 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-8dfa111e-2f92-4abb-a22b-ee8314bff7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:37.237831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 860/1711] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:37.242709 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867b4da-9d4b-4931-92b7-bb716695f1a7 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:37.258189 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa111e-2f92-4abb-a22b-ee8314bff7dc None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:37.281405 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb470625-71cc-49c3-be3b-ed25686f3d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:37.282406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb470625-71cc-49c3-be3b-ed25686f3d15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:37.285195 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb470625-71cc-49c3-be3b-ed25686f3d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:37.286312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb470625-71cc-49c3-be3b-ed25686f3d15 admin admin] PATH_INFO: `/v3/users/d4a88fb1c9cd42c498e4c71b31a50ae5` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:37.289173 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867b4da-9d4b-4931-92b7-bb716695f1a7 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:37.299975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb470625-71cc-49c3-be3b-ed25686f3d15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4a88fb1c9cd42c498e4c71b31a50ae5)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:37.306588 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bd920-74b4-4c7e-a6b0-27c656326a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:37.307237 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bd920-74b4-4c7e-a6b0-27c656326a33 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:37.307511 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bd920-74b4-4c7e-a6b0-27c656326a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:37.307781 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bd920-74b4-4c7e-a6b0-27c656326a33 admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:37.308440 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006bd920-74b4-4c7e-a6b0-27c656326a33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:12:37.308737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006bd920-74b4-4c7e-a6b0-27c656326a33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:37.310829 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb470625-71cc-49c3-be3b-ed25686f3d15 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:37.311119 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006bd920-74b4-4c7e-a6b0-27c656326a33 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:37.357316 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 861/1712] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:12:37.361075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bd920-74b4-4c7e-a6b0-27c656326a33 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:37.384934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bd920-74b4-4c7e-a6b0-27c656326a33 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:37.401795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2ea56-8fe4-4d08-8b75-1b233d747b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:37.402775 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2ea56-8fe4-4d08-8b75-1b233d747b67 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:37.403186 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2ea56-8fe4-4d08-8b75-1b233d747b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:37.403616 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2ea56-8fe4-4d08-8b75-1b233d747b67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:37.404413 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2ea56-8fe4-4d08-8b75-1b233d747b67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:37.407830 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2ea56-8fe4-4d08-8b75-1b233d747b67 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:37.458499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 862/1713] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:37.461907 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2ea56-8fe4-4d08-8b75-1b233d747b67 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:37.467965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-eb470625-71cc-49c3-be3b-ed25686f3d15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4a88fb1c9cd42c498e4c71b31a50ae5'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:37.479397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-eb470625-71cc-49c3-be3b-ed25686f3d15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4a88fb1c9cd42c498e4c71b31a50ae5'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:37.481802 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:37.494242 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2ea56-8fe4-4d08-8b75-1b233d747b67 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:37.505981 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d11e3-241b-4ced-bd09-f6647de616f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:37.506793 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d11e3-241b-4ced-bd09-f6647de616f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:37.507200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d11e3-241b-4ced-bd09-f6647de616f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:37.507957 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d11e3-241b-4ced-bd09-f6647de616f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:37.522021 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d11e3-241b-4ced-bd09-f6647de616f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:37.525195 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d11e3-241b-4ced-bd09-f6647de616f3 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:37.556431 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-eb470625-71cc-49c3-be3b-ed25686f3d15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4a88fb1c9cd42c498e4c71b31a50ae5'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:37.557258 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-eb470625-71cc-49c3-be3b-ed25686f3d15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4a88fb1c9cd42c498e4c71b31a50ae5'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:37.599036 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-eb470625-71cc-49c3-be3b-ed25686f3d15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4a88fb1c9cd42c498e4c71b31a50ae5'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:37.608130 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 863/1714] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:12:37 2022] POST /identity/v3/projects => generated 379 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:37.609082 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d11e3-241b-4ced-bd09-f6647de616f3 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:37.611249 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-d97d11e3-241b-4ced-bd09-f6647de616f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:37.617396 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 852/1715] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:12:37 2022] DELETE /identity/v3/users/d4a88fb1c9cd42c498e4c71b31a50ae5 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:37.618632 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb470625-71cc-49c3-be3b-ed25686f3d15 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:37.625196 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-eb470625-71cc-49c3-be3b-ed25686f3d15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:37.772242 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d11e3-241b-4ced-bd09-f6647de616f3 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:37.841526 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:37.850712 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb470625-71cc-49c3-be3b-ed25686f3d15 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:37.859610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e78c66-e740-4ade-84f4-d84befd39713 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:37.861214 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e78c66-e740-4ade-84f4-d84befd39713 service nova] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:37.861639 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e78c66-e740-4ade-84f4-d84befd39713 service nova] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:37.862067 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e78c66-e740-4ade-84f4-d84befd39713 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:37.865064 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b0e78c66-e740-4ade-84f4-d84befd39713 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:37.907245 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3410e8c-7f32-4ef6-8efc-3ec161879990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:37.908852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3410e8c-7f32-4ef6-8efc-3ec161879990 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:37.909277 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3410e8c-7f32-4ef6-8efc-3ec161879990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:37.910844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3410e8c-7f32-4ef6-8efc-3ec161879990 admin admin] PATH_INFO: `/v3/projects/01d166e17c174e74ac0f1203c36c12c7/users/3cfc2548e9c84b638b9cf90a833f0f35/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:37.947665 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3410e8c-7f32-4ef6-8efc-3ec161879990 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01d166e17c174e74ac0f1203c36c12c7, user_id=3cfc2548e9c84b638b9cf90a833f0f35, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:37.953721 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3410e8c-7f32-4ef6-8efc-3ec161879990 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:37.993181 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 864/1716] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:37 2022] PUT /identity/v3/projects/01d166e17c174e74ac0f1203c36c12c7/users/3cfc2548e9c84b638b9cf90a833f0f35/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:37.993864 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3410e8c-7f32-4ef6-8efc-3ec161879990 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:37.995544 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-a3410e8c-7f32-4ef6-8efc-3ec161879990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:38.036275 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e78c66-e740-4ade-84f4-d84befd39713 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:38.037983 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e78c66-e740-4ade-84f4-d84befd39713 service nova] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:38.057062 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 853/1717] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:38.072327 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed058c-2411-4680-b406-805d54a96038 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:38.075509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed058c-2411-4680-b406-805d54a96038 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:38.075966 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed058c-2411-4680-b406-805d54a96038 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:38.078479 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 854/1718] 10.222.0.44 () {54 vars in 912 bytes} [Sun Sep 11 15:12:38 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:12:38.082849 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed058c-2411-4680-b406-805d54a96038 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:38.100391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-a9ed058c-2411-4680-b406-805d54a96038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:38.137912 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3410e8c-7f32-4ef6-8efc-3ec161879990 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:38.152258 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455cdf8-b6df-4e95-ae90-8a05a5b09e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:38.152929 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455cdf8-b6df-4e95-ae90-8a05a5b09e8a admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:38.153235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455cdf8-b6df-4e95-ae90-8a05a5b09e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:38.153508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455cdf8-b6df-4e95-ae90-8a05a5b09e8a admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:38.154158 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1455cdf8-b6df-4e95-ae90-8a05a5b09e8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:38.157934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1455cdf8-b6df-4e95-ae90-8a05a5b09e8a admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:38.211278 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 865/1719] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:12:37 2022] POST /identity/v3/users => generated 290 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:38.211278 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d06df3-3beb-4ff5-a665-b14468b86b46 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:38.211278 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d06df3-3beb-4ff5-a665-b14468b86b46 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:38.211278 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d06df3-3beb-4ff5-a665-b14468b86b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:38.236599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed058c-2411-4680-b406-805d54a96038 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:38.240105 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:38.241731 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33cb54-4a56-43bf-91dc-06f1f8d23fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:38.243087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33cb54-4a56-43bf-91dc-06f1f8d23fb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:38.245856 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33cb54-4a56-43bf-91dc-06f1f8d23fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:38.246262 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33cb54-4a56-43bf-91dc-06f1f8d23fb2 admin admin] PATH_INFO: `/v3/projects/939bc03fc3a14c998b288d6a07c0ec2d` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:38.249673 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b33cb54-4a56-43bf-91dc-06f1f8d23fb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=939bc03fc3a14c998b288d6a07c0ec2d)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:38.252057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b33cb54-4a56-43bf-91dc-06f1f8d23fb2 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:38.303252 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-30d06df3-3beb-4ff5-a665-b14468b86b46 None None] MFA Rules not processed for user `3cfc2548e9c84b638b9cf90a833f0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:38.304742 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-30d06df3-3beb-4ff5-a665-b14468b86b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:38.335067 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-0b33cb54-4a56-43bf-91dc-06f1f8d23fb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '939bc03fc3a14c998b288d6a07c0ec2d', 'request_id': 'req-0b33cb54-4a56-43bf-91dc-06f1f8d23fb2'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:38.353622 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-0b33cb54-4a56-43bf-91dc-06f1f8d23fb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '939bc03fc3a14c998b288d6a07c0ec2d', 'request_id': 'req-0b33cb54-4a56-43bf-91dc-06f1f8d23fb2'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:38.362830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-0b33cb54-4a56-43bf-91dc-06f1f8d23fb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '939bc03fc3a14c998b288d6a07c0ec2d', 'request_id': 'req-0b33cb54-4a56-43bf-91dc-06f1f8d23fb2'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:38.378017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-0b33cb54-4a56-43bf-91dc-06f1f8d23fb2 admin admin] The token cache is being invalidate because project 939bc03fc3a14c998b288d6a07c0ec2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:12:38.378631 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-0b33cb54-4a56-43bf-91dc-06f1f8d23fb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:38.379393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-0b33cb54-4a56-43bf-91dc-06f1f8d23fb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:38.385568 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 855/1720] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:12:38 2022] DELETE /identity/v3/projects/939bc03fc3a14c998b288d6a07c0ec2d => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:38.386921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33cb54-4a56-43bf-91dc-06f1f8d23fb2 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:38.389062 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-0b33cb54-4a56-43bf-91dc-06f1f8d23fb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:38.452025 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:38.490507 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 866/1721] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:38.492057 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852cfda-2fdc-446f-9093-7e1483d3ecb5 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:38.497380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852cfda-2fdc-446f-9093-7e1483d3ecb5 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:38.497380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852cfda-2fdc-446f-9093-7e1483d3ecb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:38.534575 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33cb54-4a56-43bf-91dc-06f1f8d23fb2 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:38.553728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c055e-2e6c-4f77-80a6-f5158741108e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:38.556395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c055e-2e6c-4f77-80a6-f5158741108e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:38.556821 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c055e-2e6c-4f77-80a6-f5158741108e admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:38.557977 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c055e-2e6c-4f77-80a6-f5158741108e admin admin] PATH_INFO: `/v3/projects/ae744511b5684cbcb0dc51495d12e666` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:38.566136 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c055e-2e6c-4f77-80a6-f5158741108e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae744511b5684cbcb0dc51495d12e666)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:38.577800 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c055e-2e6c-4f77-80a6-f5158741108e admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:38.610467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-2852cfda-2fdc-446f-9093-7e1483d3ecb5 None None] MFA Rules not processed for user `697b73f182e6419a84e0c8c6b0fc0c06`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:38.611709 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-2852cfda-2fdc-446f-9093-7e1483d3ecb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:38.734510 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-811c055e-2e6c-4f77-80a6-f5158741108e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae744511b5684cbcb0dc51495d12e666', 'request_id': 'req-811c055e-2e6c-4f77-80a6-f5158741108e'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:38.752258 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-811c055e-2e6c-4f77-80a6-f5158741108e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae744511b5684cbcb0dc51495d12e666', 'request_id': 'req-811c055e-2e6c-4f77-80a6-f5158741108e'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:38.762670 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-811c055e-2e6c-4f77-80a6-f5158741108e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae744511b5684cbcb0dc51495d12e666', 'request_id': 'req-811c055e-2e6c-4f77-80a6-f5158741108e'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:38.773669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 867/1722] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 15:12:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:38.776413 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852cfda-2fdc-446f-9093-7e1483d3ecb5 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:38.781793 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-2852cfda-2fdc-446f-9093-7e1483d3ecb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:38.794695 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-811c055e-2e6c-4f77-80a6-f5158741108e admin admin] The token cache is being invalidate because project ae744511b5684cbcb0dc51495d12e666 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:12:38.795016 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-811c055e-2e6c-4f77-80a6-f5158741108e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:38.795951 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-811c055e-2e6c-4f77-80a6-f5158741108e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:38.801392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 856/1723] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:12:38 2022] DELETE /identity/v3/projects/ae744511b5684cbcb0dc51495d12e666 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:38.804895 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75866e-9453-49ab-bce2-5e1ea0f04170 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:38.805210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75866e-9453-49ab-bce2-5e1ea0f04170 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:38.805500 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75866e-9453-49ab-bce2-5e1ea0f04170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:38.821844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852cfda-2fdc-446f-9093-7e1483d3ecb5 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:38.846526 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-8e75866e-9453-49ab-bce2-5e1ea0f04170 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:38.856481 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-8e75866e-9453-49ab-bce2-5e1ea0f04170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:38.957806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c28070-dbc1-4d00-bafe-a49c1bd2b2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:38.958529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c28070-dbc1-4d00-bafe-a49c1bd2b2c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:38.959076 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c28070-dbc1-4d00-bafe-a49c1bd2b2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:38.959406 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c28070-dbc1-4d00-bafe-a49c1bd2b2c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:38.960131 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c28070-dbc1-4d00-bafe-a49c1bd2b2c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:38.962564 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c28070-dbc1-4d00-bafe-a49c1bd2b2c4 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:38.984556 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 857/1724] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:38.985690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75866e-9453-49ab-bce2-5e1ea0f04170 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:38.987124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-8e75866e-9453-49ab-bce2-5e1ea0f04170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:39.002335 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 868/1725] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:39.003861 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c28070-dbc1-4d00-bafe-a49c1bd2b2c4 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:39.009086 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-a2c28070-dbc1-4d00-bafe-a49c1bd2b2c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:39.011370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75866e-9453-49ab-bce2-5e1ea0f04170 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:39.027256 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54721988-760c-4a15-b673-7e7e3a225212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:39.029120 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54721988-760c-4a15-b673-7e7e3a225212 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:39.030398 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54721988-760c-4a15-b673-7e7e3a225212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:39.030779 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54721988-760c-4a15-b673-7e7e3a225212 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:39.034479 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54721988-760c-4a15-b673-7e7e3a225212 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:39.038322 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c28070-dbc1-4d00-bafe-a49c1bd2b2c4 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:39.044158 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289d7af-7b58-4b2a-86d2-eea87a58968a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:39.045027 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289d7af-7b58-4b2a-86d2-eea87a58968a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:39.045525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289d7af-7b58-4b2a-86d2-eea87a58968a admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:39.045951 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289d7af-7b58-4b2a-86d2-eea87a58968a admin admin] PATH_INFO: `/v3/users/caedfce7f45a4a7483cc3cb75d9f73f6` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:39.047454 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289d7af-7b58-4b2a-86d2-eea87a58968a admin admin] RBAC: Authorizing `identity:delete_user(user_id=caedfce7f45a4a7483cc3cb75d9f73f6)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:39.049612 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289d7af-7b58-4b2a-86d2-eea87a58968a admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:39.055693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54721988-760c-4a15-b673-7e7e3a225212 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:39.111421 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:39.141656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 858/1726] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:12:38 2022] POST /identity/v3/projects => generated 365 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:39.145436 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54721988-760c-4a15-b673-7e7e3a225212 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:39.151422 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-54721988-760c-4a15-b673-7e7e3a225212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:39.165270 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-4289d7af-7b58-4b2a-86d2-eea87a58968a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caedfce7f45a4a7483cc3cb75d9f73f6'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:39.180881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-4289d7af-7b58-4b2a-86d2-eea87a58968a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caedfce7f45a4a7483cc3cb75d9f73f6'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:39.212597 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-4289d7af-7b58-4b2a-86d2-eea87a58968a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caedfce7f45a4a7483cc3cb75d9f73f6'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:39.213521 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-4289d7af-7b58-4b2a-86d2-eea87a58968a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caedfce7f45a4a7483cc3cb75d9f73f6'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:39.227174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-4289d7af-7b58-4b2a-86d2-eea87a58968a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caedfce7f45a4a7483cc3cb75d9f73f6'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:39.243656 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 869/1727] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:12:39 2022] DELETE /identity/v3/users/caedfce7f45a4a7483cc3cb75d9f73f6 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:39.245887 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a640b2-2b67-4470-b8ce-e5d53d5ba873 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:39.246533 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a640b2-2b67-4470-b8ce-e5d53d5ba873 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:39.246985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a640b2-2b67-4470-b8ce-e5d53d5ba873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:39.280854 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-30a640b2-2b67-4470-b8ce-e5d53d5ba873 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:39.282339 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-30a640b2-2b67-4470-b8ce-e5d53d5ba873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:39.306614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:39.344859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54721988-760c-4a15-b673-7e7e3a225212 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:39.414410 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 870/1728] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:39.416754 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a640b2-2b67-4470-b8ce-e5d53d5ba873 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:39.421958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-30a640b2-2b67-4470-b8ce-e5d53d5ba873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:39.451644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a640b2-2b67-4470-b8ce-e5d53d5ba873 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:39.453208 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52b557-37aa-4e5b-a48c-e46d96cf9cd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:39.454070 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52b557-37aa-4e5b-a48c-e46d96cf9cd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:39.454802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52b557-37aa-4e5b-a48c-e46d96cf9cd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:39.454802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52b557-37aa-4e5b-a48c-e46d96cf9cd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:39.456886 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-3f52b557-37aa-4e5b-a48c-e46d96cf9cd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:39.462878 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0df9435-e9b0-47e4-9233-f459eb39d85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:39.463703 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df9435-e9b0-47e4-9233-f459eb39d85a admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:39.464356 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df9435-e9b0-47e4-9233-f459eb39d85a admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:39.469296 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df9435-e9b0-47e4-9233-f459eb39d85a admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:39.472694 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0df9435-e9b0-47e4-9233-f459eb39d85a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:39.474778 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0df9435-e9b0-47e4-9233-f459eb39d85a admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:39.487823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52b557-37aa-4e5b-a48c-e46d96cf9cd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:39.491063 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52b557-37aa-4e5b-a48c-e46d96cf9cd6 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:39.508233 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 871/1729] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:39.509899 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0df9435-e9b0-47e4-9233-f459eb39d85a admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:39.514032 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 859/1730] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:39.515062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-a0df9435-e9b0-47e4-9233-f459eb39d85a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:39.517470 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52b557-37aa-4e5b-a48c-e46d96cf9cd6 service neutron] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:39.542149 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52b557-37aa-4e5b-a48c-e46d96cf9cd6 service neutron] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:39.542706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0df9435-e9b0-47e4-9233-f459eb39d85a admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:39.552103 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82912e-23cc-469e-b6d1-fd37fc96e0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:39.553274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916abfe-c1ca-444c-b71e-de9f8beedda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:39.554142 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916abfe-c1ca-444c-b71e-de9f8beedda8 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:39.555022 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82912e-23cc-469e-b6d1-fd37fc96e0e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:39.555499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916abfe-c1ca-444c-b71e-de9f8beedda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:39.555990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82912e-23cc-469e-b6d1-fd37fc96e0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:39.556474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916abfe-c1ca-444c-b71e-de9f8beedda8 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:39.556936 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82912e-23cc-469e-b6d1-fd37fc96e0e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:39.558079 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82912e-23cc-469e-b6d1-fd37fc96e0e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:12:39.558796 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916abfe-c1ca-444c-b71e-de9f8beedda8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:39.559322 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82912e-23cc-469e-b6d1-fd37fc96e0e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:39.561962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82912e-23cc-469e-b6d1-fd37fc96e0e5 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:39.563545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916abfe-c1ca-444c-b71e-de9f8beedda8 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:39.590370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 860/1731] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:12:39.593729 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11297e2-ae4b-49a1-b23a-43b3b73f2c52 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:39.595135 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11297e2-ae4b-49a1-b23a-43b3b73f2c52 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:39.595617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11297e2-ae4b-49a1-b23a-43b3b73f2c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:39.616130 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 872/1732] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:12:39 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:39.618455 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916abfe-c1ca-444c-b71e-de9f8beedda8 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:39.643734 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916abfe-c1ca-444c-b71e-de9f8beedda8 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:39.656706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8cfd1-9fea-4aae-a06d-40566f3f5f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:39.658206 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8cfd1-9fea-4aae-a06d-40566f3f5f93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:39.659226 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8cfd1-9fea-4aae-a06d-40566f3f5f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:39.659611 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8cfd1-9fea-4aae-a06d-40566f3f5f93 admin admin] PATH_INFO: `/v3/projects/80b9bbfd704c48c292a345ba08af93b9/users/3c9aebb52950445d81c18ce551fe06b3/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:39.674228 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-e11297e2-ae4b-49a1-b23a-43b3b73f2c52 None None] MFA Rules not processed for user `2bd47bf9aa4142b1b7ec8d02e8661578`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:39.675583 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-e11297e2-ae4b-49a1-b23a-43b3b73f2c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:39.698848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8cfd1-9fea-4aae-a06d-40566f3f5f93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80b9bbfd704c48c292a345ba08af93b9, user_id=3c9aebb52950445d81c18ce551fe06b3, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:39.703568 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8cfd1-9fea-4aae-a06d-40566f3f5f93 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:39.759524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 873/1733] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:39 2022] PUT /identity/v3/projects/80b9bbfd704c48c292a345ba08af93b9/users/3c9aebb52950445d81c18ce551fe06b3/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:39.759524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8cfd1-9fea-4aae-a06d-40566f3f5f93 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:39.929896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 861/1734] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:39 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:39.929896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11297e2-ae4b-49a1-b23a-43b3b73f2c52 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:39.942585 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:DELETE=8,keystone:SELECT=87,keystone:INSERT=5 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:39.947767 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11297e2-ae4b-49a1-b23a-43b3b73f2c52 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:40.002248 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8cfd1-9fea-4aae-a06d-40566f3f5f93 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:40.010370 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8bd7ea-2bca-410a-b909-519a2ca84a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:40.010999 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8bd7ea-2bca-410a-b909-519a2ca84a46 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:40.011312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8bd7ea-2bca-410a-b909-519a2ca84a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:40.011535 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8bd7ea-2bca-410a-b909-519a2ca84a46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:40.012343 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8bd7ea-2bca-410a-b909-519a2ca84a46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:40.014401 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8bd7ea-2bca-410a-b909-519a2ca84a46 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:40.074873 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 874/1735] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:12:39 2022] POST /identity/v3/projects => generated 387 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:40.076723 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f739f-501b-4232-bd76-3809839d2afc None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:40.077008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f739f-501b-4232-bd76-3809839d2afc None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:40.077304 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f739f-501b-4232-bd76-3809839d2afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:40.081013 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fdbf79-92dc-4eed-a835-dc77aba05717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:40.081914 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdbf79-92dc-4eed-a835-dc77aba05717 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:40.082367 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdbf79-92dc-4eed-a835-dc77aba05717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:40.083042 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdbf79-92dc-4eed-a835-dc77aba05717 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:40.083883 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fdbf79-92dc-4eed-a835-dc77aba05717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:40.086296 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fdbf79-92dc-4eed-a835-dc77aba05717 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:40.099906 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 862/1736] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:40.103363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fdbf79-92dc-4eed-a835-dc77aba05717 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:40.105675 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-35fdbf79-92dc-4eed-a835-dc77aba05717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:40.130448 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fdbf79-92dc-4eed-a835-dc77aba05717 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:40.135613 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-fb6f739f-501b-4232-bd76-3809839d2afc None None] MFA Rules not processed for user `3c9aebb52950445d81c18ce551fe06b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:40.137810 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-fb6f739f-501b-4232-bd76-3809839d2afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:40.155644 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:40.283550 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e6851-e9b8-4f97-b69a-af39edd649e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:40.286265 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e6851-e9b8-4f97-b69a-af39edd649e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:40.286544 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e6851-e9b8-4f97-b69a-af39edd649e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:40.287192 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e6851-e9b8-4f97-b69a-af39edd649e3 admin admin] PATH_INFO: `/v3/projects/345261108b45454eb064fff7a2fb75a9` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:40.291985 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9e6851-e9b8-4f97-b69a-af39edd649e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=345261108b45454eb064fff7a2fb75a9)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:40.297084 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9e6851-e9b8-4f97-b69a-af39edd649e3 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:40.318522 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 875/1737] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:40.322528 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d174e-93f8-4d65-965b-3d23c93353cb None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:40.322819 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d174e-93f8-4d65-965b-3d23c93353cb None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:40.325531 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d174e-93f8-4d65-965b-3d23c93353cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:40.378166 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-9f9d174e-93f8-4d65-965b-3d23c93353cb None None] MFA Rules not processed for user `2bd47bf9aa4142b1b7ec8d02e8661578`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:40.379688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-9f9d174e-93f8-4d65-965b-3d23c93353cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:40.430003 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3d9e6851-e9b8-4f97-b69a-af39edd649e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '345261108b45454eb064fff7a2fb75a9', 'request_id': 'req-3d9e6851-e9b8-4f97-b69a-af39edd649e3'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:40.449209 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3d9e6851-e9b8-4f97-b69a-af39edd649e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '345261108b45454eb064fff7a2fb75a9', 'request_id': 'req-3d9e6851-e9b8-4f97-b69a-af39edd649e3'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:40.460154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3d9e6851-e9b8-4f97-b69a-af39edd649e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '345261108b45454eb064fff7a2fb75a9', 'request_id': 'req-3d9e6851-e9b8-4f97-b69a-af39edd649e3'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:40.491169 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3d9e6851-e9b8-4f97-b69a-af39edd649e3 admin admin] The token cache is being invalidate because project 345261108b45454eb064fff7a2fb75a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:12:40.492759 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3d9e6851-e9b8-4f97-b69a-af39edd649e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:40.494937 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3d9e6851-e9b8-4f97-b69a-af39edd649e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:40.499850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 863/1738] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:12:40 2022] DELETE /identity/v3/projects/345261108b45454eb064fff7a2fb75a9 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:40.505293 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e6851-e9b8-4f97-b69a-af39edd649e3 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:40.507273 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-3d9e6851-e9b8-4f97-b69a-af39edd649e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:40.526434 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 876/1739] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:40 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:40.527358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d174e-93f8-4d65-965b-3d23c93353cb None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:40.528778 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-9f9d174e-93f8-4d65-965b-3d23c93353cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:40.549951 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d174e-93f8-4d65-965b-3d23c93353cb None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:40.625519 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55783d0c-10ce-4a6b-aa47-a869babc8cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:40.626202 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55783d0c-10ce-4a6b-aa47-a869babc8cdf admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:40.626477 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55783d0c-10ce-4a6b-aa47-a869babc8cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:40.626744 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55783d0c-10ce-4a6b-aa47-a869babc8cdf admin admin] PATH_INFO: `/v3/projects/163c5689a40e4f76841cb11e9f68bca5/users/9635f4ef205146e5a943de36ee03377a/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:40.635403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:40.651039 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55783d0c-10ce-4a6b-aa47-a869babc8cdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=163c5689a40e4f76841cb11e9f68bca5, user_id=9635f4ef205146e5a943de36ee03377a, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:40.654450 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55783d0c-10ce-4a6b-aa47-a869babc8cdf admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:40.693557 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 877/1740] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:40 2022] PUT /identity/v3/projects/163c5689a40e4f76841cb11e9f68bca5/users/9635f4ef205146e5a943de36ee03377a/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:40.695687 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55783d0c-10ce-4a6b-aa47-a869babc8cdf admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:40.705103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-55783d0c-10ce-4a6b-aa47-a869babc8cdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:40.713321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e6851-e9b8-4f97-b69a-af39edd649e3 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:40.814713 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:40.861492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4759310-a768-40d0-aa3f-23d6498a3663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:40.862122 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4759310-a768-40d0-aa3f-23d6498a3663 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:40.862395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4759310-a768-40d0-aa3f-23d6498a3663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:40.862658 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4759310-a768-40d0-aa3f-23d6498a3663 admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:40.863293 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4759310-a768-40d0-aa3f-23d6498a3663 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:40.865250 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4759310-a768-40d0-aa3f-23d6498a3663 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:40.885006 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55783d0c-10ce-4a6b-aa47-a869babc8cdf admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:40.895026 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91448fc5-97ec-4794-9ac1-19baa82fef62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:40.895849 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91448fc5-97ec-4794-9ac1-19baa82fef62 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:40.896121 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91448fc5-97ec-4794-9ac1-19baa82fef62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:40.896475 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91448fc5-97ec-4794-9ac1-19baa82fef62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:40.899894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91448fc5-97ec-4794-9ac1-19baa82fef62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:40.904160 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91448fc5-97ec-4794-9ac1-19baa82fef62 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:40.914244 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 864/1741] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:12:40 2022] POST /identity/v3/users => generated 294 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:40.915377 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ad122-7d8f-4146-90b5-1f68a96a8634 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:40.915598 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ad122-7d8f-4146-90b5-1f68a96a8634 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:40.915871 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ad122-7d8f-4146-90b5-1f68a96a8634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:40.957778 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-c86ad122-7d8f-4146-90b5-1f68a96a8634 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:40.959027 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c86ad122-7d8f-4146-90b5-1f68a96a8634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:40.961384 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 878/1742] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:12:40 2022] POST /identity/v3/projects => generated 381 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:40.961950 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91448fc5-97ec-4794-9ac1-19baa82fef62 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:40.965927 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-91448fc5-97ec-4794-9ac1-19baa82fef62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:41.101188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 865/1743] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:41.101188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ad122-7d8f-4146-90b5-1f68a96a8634 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:41.115522 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ad122-7d8f-4146-90b5-1f68a96a8634 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:41.123255 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91448fc5-97ec-4794-9ac1-19baa82fef62 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:41.125877 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d650c0-af19-4941-8c9e-036789f71f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:41.126517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d650c0-af19-4941-8c9e-036789f71f85 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:41.126786 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d650c0-af19-4941-8c9e-036789f71f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:41.127094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d650c0-af19-4941-8c9e-036789f71f85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:41.127700 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d650c0-af19-4941-8c9e-036789f71f85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:41.130786 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d650c0-af19-4941-8c9e-036789f71f85 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:41.136783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61e605-8953-4064-a5a2-d0c200a802f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:41.137777 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61e605-8953-4064-a5a2-d0c200a802f4 service nova] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:41.139507 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61e605-8953-4064-a5a2-d0c200a802f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:41.139968 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61e605-8953-4064-a5a2-d0c200a802f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:41.142378 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 866/1744] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:41.143184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d650c0-af19-4941-8c9e-036789f71f85 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:41.145177 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-68d650c0-af19-4941-8c9e-036789f71f85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:41.146132 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-ec61e605-8953-4064-a5a2-d0c200a802f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:41.248094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d650c0-af19-4941-8c9e-036789f71f85 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:41.254859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e7bc6-81e9-497a-9a6b-a0a80fef42ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead06362f8a2444bbfe1405cd602f1c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:41.255557 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e7bc6-81e9-497a-9a6b-a0a80fef42ec service placement] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:41.255850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e7bc6-81e9-497a-9a6b-a0a80fef42ec service placement] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:41.256253 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e7bc6-81e9-497a-9a6b-a0a80fef42ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:41.258379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-788e7bc6-81e9-497a-9a6b-a0a80fef42ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:41.282696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e7bc6-81e9-497a-9a6b-a0a80fef42ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:41.285007 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e7bc6-81e9-497a-9a6b-a0a80fef42ec service placement] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:41.295422 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec61e605-8953-4064-a5a2-d0c200a802f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:41.297506 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec61e605-8953-4064-a5a2-d0c200a802f4 service nova] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:41.310504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 867/1745] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:41.311395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e7bc6-81e9-497a-9a6b-a0a80fef42ec service placement] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:41.322303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 879/1746] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:40 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:41.323174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61e605-8953-4064-a5a2-d0c200a802f4 service nova] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:41.333456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e7bc6-81e9-497a-9a6b-a0a80fef42ec service placement] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:41.346902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2164b-7516-4cd9-bdc9-2916bc10130f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:41.349122 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2164b-7516-4cd9-bdc9-2916bc10130f admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:41.349451 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2164b-7516-4cd9-bdc9-2916bc10130f admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:41.349689 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2164b-7516-4cd9-bdc9-2916bc10130f admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:41.350149 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61e605-8953-4064-a5a2-d0c200a802f4 service nova] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:41.350347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2164b-7516-4cd9-bdc9-2916bc10130f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:41.356136 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2164b-7516-4cd9-bdc9-2916bc10130f admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:41.356391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8550734-f6fb-4fb0-8100-179178f360e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:41.356775 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8550734-f6fb-4fb0-8100-179178f360e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:41.357033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8550734-f6fb-4fb0-8100-179178f360e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:41.357327 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8550734-f6fb-4fb0-8100-179178f360e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:41.357968 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8550734-f6fb-4fb0-8100-179178f360e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:41.360877 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8550734-f6fb-4fb0-8100-179178f360e1 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:41.382185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:41.385219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 868/1747] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:41.393838 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38115103-7689-46d3-9849-550d3ea4cfa8 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:41.394666 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38115103-7689-46d3-9849-550d3ea4cfa8 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:41.394978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38115103-7689-46d3-9849-550d3ea4cfa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:41.416842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 880/1748] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:12:41 2022] POST /identity/v3/users => generated 297 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:41.416842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8550734-f6fb-4fb0-8100-179178f360e1 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:41.466997 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8550734-f6fb-4fb0-8100-179178f360e1 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:41.468325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-38115103-7689-46d3-9849-550d3ea4cfa8 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:41.472333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-38115103-7689-46d3-9849-550d3ea4cfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:41.481990 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a0f16-2e29-44fd-a201-21692b4e252d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:41.484365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a0f16-2e29-44fd-a201-21692b4e252d admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:41.484670 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a0f16-2e29-44fd-a201-21692b4e252d admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:41.485508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a0f16-2e29-44fd-a201-21692b4e252d admin admin] PATH_INFO: `/v3/projects/163c5689a40e4f76841cb11e9f68bca5/users/9635f4ef205146e5a943de36ee03377a/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:41.490943 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a0f16-2e29-44fd-a201-21692b4e252d admin admin] RBAC: Authorizing `identity:create_grant(project_id=163c5689a40e4f76841cb11e9f68bca5, user_id=9635f4ef205146e5a943de36ee03377a, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:41.501170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 869/1749] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:41.503385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a0f16-2e29-44fd-a201-21692b4e252d admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:41.504317 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38115103-7689-46d3-9849-550d3ea4cfa8 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:41.526733 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38115103-7689-46d3-9849-550d3ea4cfa8 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:41.543573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 881/1750] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:41 2022] PUT /identity/v3/projects/163c5689a40e4f76841cb11e9f68bca5/users/9635f4ef205146e5a943de36ee03377a/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:41.547307 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a0f16-2e29-44fd-a201-21692b4e252d admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:41.669492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:41.724013 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf8948-e603-4f41-99a8-796bcc17640f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:41.724654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf8948-e603-4f41-99a8-796bcc17640f admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:41.724960 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf8948-e603-4f41-99a8-796bcc17640f admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:41.725638 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf8948-e603-4f41-99a8-796bcc17640f admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:41.729344 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf8948-e603-4f41-99a8-796bcc17640f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:41.735457 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf8948-e603-4f41-99a8-796bcc17640f admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:41.752214 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a0f16-2e29-44fd-a201-21692b4e252d admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:41.766656 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8dbdf-d1b1-4b54-b014-dedcf41744aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:41.766857 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 870/1751] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:41.768074 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8dbdf-d1b1-4b54-b014-dedcf41744aa admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:41.768518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf8948-e603-4f41-99a8-796bcc17640f admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:41.769460 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8dbdf-d1b1-4b54-b014-dedcf41744aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:41.769737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8dbdf-d1b1-4b54-b014-dedcf41744aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:41.772485 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8dbdf-d1b1-4b54-b014-dedcf41744aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:41.775575 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8dbdf-d1b1-4b54-b014-dedcf41744aa admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:41.793000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf8948-e603-4f41-99a8-796bcc17640f admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:41.803291 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 882/1752] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:41.805206 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3376d9-3495-4a38-a3d0-81b790e9cddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:41.805902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3376d9-3495-4a38-a3d0-81b790e9cddc admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:41.806690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f22349-8a6f-45a5-ad04-21b1af823f4c None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:41.806984 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3376d9-3495-4a38-a3d0-81b790e9cddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:41.807363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3376d9-3495-4a38-a3d0-81b790e9cddc admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:41.808605 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f22349-8a6f-45a5-ad04-21b1af823f4c None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:41.808884 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f22349-8a6f-45a5-ad04-21b1af823f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:41.810079 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3376d9-3495-4a38-a3d0-81b790e9cddc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:12:41.810526 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3376d9-3495-4a38-a3d0-81b790e9cddc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:41.813707 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3376d9-3495-4a38-a3d0-81b790e9cddc admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:41.851369 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 871/1753] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:12:41.852325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3376d9-3495-4a38-a3d0-81b790e9cddc admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:41.864779 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-14f22349-8a6f-45a5-ad04-21b1af823f4c None None] MFA Rules not processed for user `9635f4ef205146e5a943de36ee03377a`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:41.869240 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-14f22349-8a6f-45a5-ad04-21b1af823f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:41.872617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3376d9-3495-4a38-a3d0-81b790e9cddc admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:41.884757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d5371-0338-48de-b203-8c53239ddf86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:41.885537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d5371-0338-48de-b203-8c53239ddf86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:41.885814 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d5371-0338-48de-b203-8c53239ddf86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:41.886134 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d5371-0338-48de-b203-8c53239ddf86 admin admin] PATH_INFO: `/v3/projects/881ee6aa169c4dc7bb8fad51170e7e30/users/794f74d3cc14458aaa28621128f699b6/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:41.923571 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d5371-0338-48de-b203-8c53239ddf86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=881ee6aa169c4dc7bb8fad51170e7e30, user_id=794f74d3cc14458aaa28621128f699b6, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:41.926696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d5371-0338-48de-b203-8c53239ddf86 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:41.954942 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 872/1754] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:41 2022] PUT /identity/v3/projects/881ee6aa169c4dc7bb8fad51170e7e30/users/794f74d3cc14458aaa28621128f699b6/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:41.955925 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d5371-0338-48de-b203-8c53239ddf86 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:42.085303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 883/1755] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:41 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:42.085303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f22349-8a6f-45a5-ad04-21b1af823f4c None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:42.103618 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f22349-8a6f-45a5-ad04-21b1af823f4c None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:42.110830 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351f417-7710-4f37-a5aa-a041edc5ae71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:42.110960 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351f417-7710-4f37-a5aa-a041edc5ae71 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:42.111354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351f417-7710-4f37-a5aa-a041edc5ae71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:42.111615 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351f417-7710-4f37-a5aa-a041edc5ae71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:42.112403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6351f417-7710-4f37-a5aa-a041edc5ae71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:42.120140 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6351f417-7710-4f37-a5aa-a041edc5ae71 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:42.156231 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 884/1756] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:12:42 2022] POST /identity/v3/projects => generated 369 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:42.158127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34315d-55e1-48dd-ac3a-1127dfc807b5 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:42.158127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34315d-55e1-48dd-ac3a-1127dfc807b5 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:42.158309 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d5371-0338-48de-b203-8c53239ddf86 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:42.158688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34315d-55e1-48dd-ac3a-1127dfc807b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:42.205916 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-8b34315d-55e1-48dd-ac3a-1127dfc807b5 None None] MFA Rules not processed for user `794f74d3cc14458aaa28621128f699b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:42.210026 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-8b34315d-55e1-48dd-ac3a-1127dfc807b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:42.263823 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:42.370170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 885/1757] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:42.371345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34315d-55e1-48dd-ac3a-1127dfc807b5 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:42.373330 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-8b34315d-55e1-48dd-ac3a-1127dfc807b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:42.417911 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2cfddb-4844-4133-aeae-a55bc033b759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:42.418909 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cfddb-4844-4133-aeae-a55bc033b759 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:42.419392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cfddb-4844-4133-aeae-a55bc033b759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:42.419896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cfddb-4844-4133-aeae-a55bc033b759 admin admin] PATH_INFO: `/v3/projects/bdcdd9681fe344c886f6afbf5b93bf12/users/3cf7193cea0344169df1b5d549f02a83/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:42.457272 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2cfddb-4844-4133-aeae-a55bc033b759 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdcdd9681fe344c886f6afbf5b93bf12, user_id=3cf7193cea0344169df1b5d549f02a83, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:42.462381 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2cfddb-4844-4133-aeae-a55bc033b759 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:42.472193 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34315d-55e1-48dd-ac3a-1127dfc807b5 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:42.477508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194cd073-ae01-420a-ae29-fea4b7077966 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:42.478545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194cd073-ae01-420a-ae29-fea4b7077966 service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:42.479047 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194cd073-ae01-420a-ae29-fea4b7077966 service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:42.479539 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194cd073-ae01-420a-ae29-fea4b7077966 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:42.495284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 873/1758] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:41 2022] PUT /identity/v3/projects/bdcdd9681fe344c886f6afbf5b93bf12/users/3cf7193cea0344169df1b5d549f02a83/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:42.496775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2cfddb-4844-4133-aeae-a55bc033b759 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:42.569242 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:42.582547 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194cd073-ae01-420a-ae29-fea4b7077966 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:42.607317 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194cd073-ae01-420a-ae29-fea4b7077966 service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:42.651476 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 886/1759] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:42 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:42.655280 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8fdbf-a813-436c-8d53-afb3ef980b14 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:42.655581 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8fdbf-a813-436c-8d53-afb3ef980b14 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:42.655858 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8fdbf-a813-436c-8d53-afb3ef980b14 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:42.657867 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2cfddb-4844-4133-aeae-a55bc033b759 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:42.673639 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04912fe6-0a03-4c4d-aaed-967845b81f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:42.674580 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04912fe6-0a03-4c4d-aaed-967845b81f60 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:42.674709 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 887/1760] 10.222.0.44 () {54 vars in 912 bytes} [Sun Sep 11 15:12:42 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:12:42.675258 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8fdbf-a813-436c-8d53-afb3ef980b14 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:42.681292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04912fe6-0a03-4c4d-aaed-967845b81f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:42.681771 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04912fe6-0a03-4c4d-aaed-967845b81f60 admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:42.683142 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04912fe6-0a03-4c4d-aaed-967845b81f60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:42.687431 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04912fe6-0a03-4c4d-aaed-967845b81f60 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:42.744520 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 874/1761] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:12:42 2022] POST /identity/v3/users => generated 285 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:42.745923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8fdbf-a813-436c-8d53-afb3ef980b14 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:42.755306 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04912fe6-0a03-4c4d-aaed-967845b81f60 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:42.768978 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8f274-9e2d-4660-ae92-2a3ccdf543df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:42.773869 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8f274-9e2d-4660-ae92-2a3ccdf543df admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:42.774738 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8f274-9e2d-4660-ae92-2a3ccdf543df admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:42.775219 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8f274-9e2d-4660-ae92-2a3ccdf543df admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:42.776142 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8f274-9e2d-4660-ae92-2a3ccdf543df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:42.778766 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8f274-9e2d-4660-ae92-2a3ccdf543df admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:42.779875 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04912fe6-0a03-4c4d-aaed-967845b81f60 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:42.797323 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3f6e0-340f-43a8-be27-8426460f5654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:42.810913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3f6e0-340f-43a8-be27-8426460f5654 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:42.812987 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3f6e0-340f-43a8-be27-8426460f5654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:42.813396 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3f6e0-340f-43a8-be27-8426460f5654 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:42.814283 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b3f6e0-340f-43a8-be27-8426460f5654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:42.819268 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b3f6e0-340f-43a8-be27-8426460f5654 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:42.846679 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 888/1762] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:12:42 2022] POST /identity/v3/projects => generated 385 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:42.848253 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3fc52-9d6d-485e-a1bd-3a1db6175691 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:42.848537 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3fc52-9d6d-485e-a1bd-3a1db6175691 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:42.848808 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3fc52-9d6d-485e-a1bd-3a1db6175691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:42.849972 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 875/1763] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:42.850991 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3f6e0-340f-43a8-be27-8426460f5654 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:42.887871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-10a3fc52-9d6d-485e-a1bd-3a1db6175691 None None] MFA Rules not processed for user `697b73f182e6419a84e0c8c6b0fc0c06`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:42.889737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-10a3fc52-9d6d-485e-a1bd-3a1db6175691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:42.902608 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3f6e0-340f-43a8-be27-8426460f5654 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:43.060885 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 889/1764] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 15:12:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:43.067860 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a3fc52-9d6d-485e-a1bd-3a1db6175691 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:43.085804 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:43.151536 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3580a4d-2e85-4be9-be63-8249e0e6e655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:43.152355 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3580a4d-2e85-4be9-be63-8249e0e6e655 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:43.152511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3580a4d-2e85-4be9-be63-8249e0e6e655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:43.152774 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3580a4d-2e85-4be9-be63-8249e0e6e655 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:43.153473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3580a4d-2e85-4be9-be63-8249e0e6e655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:43.155491 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3580a4d-2e85-4be9-be63-8249e0e6e655 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:43.181454 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 876/1765] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:43.184965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3580a4d-2e85-4be9-be63-8249e0e6e655 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:43.219541 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3580a4d-2e85-4be9-be63-8249e0e6e655 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:43.228749 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0858214-1d70-490a-9266-a4bd410b501a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:43.229725 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0858214-1d70-490a-9266-a4bd410b501a admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:43.230252 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0858214-1d70-490a-9266-a4bd410b501a admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:43.230713 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0858214-1d70-490a-9266-a4bd410b501a admin admin] PATH_INFO: `/v3/projects/bdcdd9681fe344c886f6afbf5b93bf12/users/3cf7193cea0344169df1b5d549f02a83/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:43.233242 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0858214-1d70-490a-9266-a4bd410b501a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdcdd9681fe344c886f6afbf5b93bf12, user_id=3cf7193cea0344169df1b5d549f02a83, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:43.236715 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0858214-1d70-490a-9266-a4bd410b501a admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:43.300434 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 877/1766] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:43 2022] PUT /identity/v3/projects/bdcdd9681fe344c886f6afbf5b93bf12/users/3cf7193cea0344169df1b5d549f02a83/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:43.302417 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0858214-1d70-490a-9266-a4bd410b501a admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:43.311927 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f0858214-1d70-490a-9266-a4bd410b501a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:43.363901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a3fc52-9d6d-485e-a1bd-3a1db6175691 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:43.460206 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0858214-1d70-490a-9266-a4bd410b501a admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:43.471186 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ffe88-f190-40ca-aea2-ca8c28869141 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:43.471615 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ffe88-f190-40ca-aea2-ca8c28869141 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:43.471888 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ffe88-f190-40ca-aea2-ca8c28869141 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:43.472156 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ffe88-f190-40ca-aea2-ca8c28869141 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:43.506670 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886ffe88-f190-40ca-aea2-ca8c28869141 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:43.510973 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886ffe88-f190-40ca-aea2-ca8c28869141 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:43.555114 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 878/1767] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:43.556124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ffe88-f190-40ca-aea2-ca8c28869141 service neutron] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:43.579761 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d71c29-16ef-486a-b29f-38b3c5b516b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:43.582951 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d71c29-16ef-486a-b29f-38b3c5b516b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:43.583244 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d71c29-16ef-486a-b29f-38b3c5b516b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:43.583545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d71c29-16ef-486a-b29f-38b3c5b516b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:43.584367 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d71c29-16ef-486a-b29f-38b3c5b516b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:43.597825 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d71c29-16ef-486a-b29f-38b3c5b516b7 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:43.629509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:43.689175 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 890/1768] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:12:43 2022] POST /identity/v3/users => generated 293 bytes in 625 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:43.689175 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc9297d-7bdf-4654-abc3-0c03f70ba2e9 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:43.689175 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc9297d-7bdf-4654-abc3-0c03f70ba2e9 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:43.689175 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc9297d-7bdf-4654-abc3-0c03f70ba2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:43.702577 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ffe88-f190-40ca-aea2-ca8c28869141 service neutron] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:43.713335 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6b436-8224-41a5-af15-20a5c8d931c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:43.715606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6b436-8224-41a5-af15-20a5c8d931c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:43.720790 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6b436-8224-41a5-af15-20a5c8d931c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:43.732333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6b436-8224-41a5-af15-20a5c8d931c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:43.733092 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b6b436-8224-41a5-af15-20a5c8d931c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:43.734761 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b6b436-8224-41a5-af15-20a5c8d931c8 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:43.738473 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-4bc9297d-7bdf-4654-abc3-0c03f70ba2e9 None None] MFA Rules not processed for user `3cf7193cea0344169df1b5d549f02a83`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:43.739702 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-4bc9297d-7bdf-4654-abc3-0c03f70ba2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:43.764032 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 879/1769] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:43.773473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffe5d2-af04-4d0a-9c90-6988ac780579 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:43.773838 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffe5d2-af04-4d0a-9c90-6988ac780579 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:43.774028 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffe5d2-af04-4d0a-9c90-6988ac780579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:43.806017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-90ffe5d2-af04-4d0a-9c90-6988ac780579 None None] MFA Rules not processed for user `5e4f8b13b66449cf849ff9cb73938c4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:43.807917 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-90ffe5d2-af04-4d0a-9c90-6988ac780579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:43.900642 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 891/1770] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:43 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:43.901616 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc9297d-7bdf-4654-abc3-0c03f70ba2e9 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:43.926228 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc9297d-7bdf-4654-abc3-0c03f70ba2e9 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:43.930061 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 880/1771] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:43.930872 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffe5d2-af04-4d0a-9c90-6988ac780579 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:43.931053 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aace87b4-0ed1-485a-8c0b-0c537a5e4438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:43.931785 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace87b4-0ed1-485a-8c0b-0c537a5e4438 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:43.931967 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace87b4-0ed1-485a-8c0b-0c537a5e4438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:43.932232 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace87b4-0ed1-485a-8c0b-0c537a5e4438 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:43.933463 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aace87b4-0ed1-485a-8c0b-0c537a5e4438 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:43.934649 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aace87b4-0ed1-485a-8c0b-0c537a5e4438 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:43.953496 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffe5d2-af04-4d0a-9c90-6988ac780579 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:43.957062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 892/1772] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:43.957960 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aace87b4-0ed1-485a-8c0b-0c537a5e4438 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:43.966919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e8a1b5-fecd-434f-856e-ac66cc500312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:43.968354 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8a1b5-fecd-434f-856e-ac66cc500312 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:43.970255 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8a1b5-fecd-434f-856e-ac66cc500312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:43.971005 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aace87b4-0ed1-485a-8c0b-0c537a5e4438 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:43.971917 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8a1b5-fecd-434f-856e-ac66cc500312 admin admin] PATH_INFO: `/v3/projects/97eead5cc7a4450b8a3477e917d434ea/users/736f2eb232fe44f6a0a6cc9824a76e98/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:43.976991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ec080-c416-4f9e-ad59-90db2a55cf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:43.977656 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ec080-c416-4f9e-ad59-90db2a55cf77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:43.977931 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ec080-c416-4f9e-ad59-90db2a55cf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:43.978201 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ec080-c416-4f9e-ad59-90db2a55cf77 admin admin] PATH_INFO: `/v3/users/3cfc2548e9c84b638b9cf90a833f0f35` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:43.980221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1ec080-c416-4f9e-ad59-90db2a55cf77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cfc2548e9c84b638b9cf90a833f0f35)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:43.982185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1ec080-c416-4f9e-ad59-90db2a55cf77 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:44.000327 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e8a1b5-fecd-434f-856e-ac66cc500312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97eead5cc7a4450b8a3477e917d434ea, user_id=736f2eb232fe44f6a0a6cc9824a76e98, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:44.003201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e8a1b5-fecd-434f-856e-ac66cc500312 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:44.005199 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:44.040216 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 881/1773] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:43 2022] PUT /identity/v3/projects/97eead5cc7a4450b8a3477e917d434ea/users/736f2eb232fe44f6a0a6cc9824a76e98/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:44.040216 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e8a1b5-fecd-434f-856e-ac66cc500312 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:44.129661 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2b1ec080-c416-4f9e-ad59-90db2a55cf77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cfc2548e9c84b638b9cf90a833f0f35'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:44.142336 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2b1ec080-c416-4f9e-ad59-90db2a55cf77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cfc2548e9c84b638b9cf90a833f0f35'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:44.159271 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e8a1b5-fecd-434f-856e-ac66cc500312 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:44.164686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b2c73-2766-47ce-88fd-b90fa308ad51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:44.165402 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b2c73-2766-47ce-88fd-b90fa308ad51 service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:44.165709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b2c73-2766-47ce-88fd-b90fa308ad51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:44.166534 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b2c73-2766-47ce-88fd-b90fa308ad51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:44.171472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2b1ec080-c416-4f9e-ad59-90db2a55cf77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cfc2548e9c84b638b9cf90a833f0f35'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:44.172575 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2b1ec080-c416-4f9e-ad59-90db2a55cf77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cfc2548e9c84b638b9cf90a833f0f35'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:44.183468 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2b1ec080-c416-4f9e-ad59-90db2a55cf77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cfc2548e9c84b638b9cf90a833f0f35'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:44.194636 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 893/1774] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:12:43 2022] DELETE /identity/v3/users/3cfc2548e9c84b638b9cf90a833f0f35 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:44.195823 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ec080-c416-4f9e-ad59-90db2a55cf77 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:44.199449 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-2b1ec080-c416-4f9e-ad59-90db2a55cf77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:44.361678 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b2c73-2766-47ce-88fd-b90fa308ad51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:44.372263 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b2c73-2766-47ce-88fd-b90fa308ad51 service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:44.383814 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:44.405381 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ec080-c416-4f9e-ad59-90db2a55cf77 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:44.411126 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 882/1775] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:44 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:44.412611 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d7caf-cee7-484a-b325-515c9197ee7d None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:44.415830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d7caf-cee7-484a-b325-515c9197ee7d None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:44.416082 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d7caf-cee7-484a-b325-515c9197ee7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:44.418870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1886a894-085a-4fa3-8514-83eddd046503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:44.419520 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1886a894-085a-4fa3-8514-83eddd046503 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:44.419800 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1886a894-085a-4fa3-8514-83eddd046503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:44.420076 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1886a894-085a-4fa3-8514-83eddd046503 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:44.420745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1886a894-085a-4fa3-8514-83eddd046503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:44.422642 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1886a894-085a-4fa3-8514-83eddd046503 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:44.459775 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 894/1776] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:44.461380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d152ee2-5d33-4c14-a407-75fac8cb5590 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:44.461671 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d152ee2-5d33-4c14-a407-75fac8cb5590 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:44.461945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d152ee2-5d33-4c14-a407-75fac8cb5590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:44.468690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-fd9d7caf-cee7-484a-b325-515c9197ee7d None None] MFA Rules not processed for user `736f2eb232fe44f6a0a6cc9824a76e98`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:44.470006 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-fd9d7caf-cee7-484a-b325-515c9197ee7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:44.553257 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-0d152ee2-5d33-4c14-a407-75fac8cb5590 None None] MFA Rules not processed for user `c44105f2d90e469fb03f311a4cf1476f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:44.553257 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-0d152ee2-5d33-4c14-a407-75fac8cb5590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:44.613088 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 883/1777] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:44.614590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1b098-9892-4383-ba17-5e66a05dfe40 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:44.615148 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1b098-9892-4383-ba17-5e66a05dfe40 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:44.615754 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1b098-9892-4383-ba17-5e66a05dfe40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:44.686014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-b8f1b098-9892-4383-ba17-5e66a05dfe40 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:44.692198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b8f1b098-9892-4383-ba17-5e66a05dfe40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:44.713934 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 884/1778] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:44.714627 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1b098-9892-4383-ba17-5e66a05dfe40 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:44.731346 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1b098-9892-4383-ba17-5e66a05dfe40 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:44.737767 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2392a44-d5b4-45fb-a13f-de45970d2c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:44.738182 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2392a44-d5b4-45fb-a13f-de45970d2c78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:44.738458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2392a44-d5b4-45fb-a13f-de45970d2c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:44.738730 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2392a44-d5b4-45fb-a13f-de45970d2c78 admin admin] PATH_INFO: `/v3/projects/d61da770f51a492ea01d2aad14a35147/users/4061c8c2422a4fd68342f1fe6524f421/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:44.784238 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 895/1779] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:44.785911 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09332cc7-1fff-44ab-b984-6e264d4ef79a None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:44.786099 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2392a44-d5b4-45fb-a13f-de45970d2c78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d61da770f51a492ea01d2aad14a35147, user_id=4061c8c2422a4fd68342f1fe6524f421, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:44.793674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09332cc7-1fff-44ab-b984-6e264d4ef79a None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:44.795221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09332cc7-1fff-44ab-b984-6e264d4ef79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:44.802118 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2392a44-d5b4-45fb-a13f-de45970d2c78 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:44.858108 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 885/1780] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:44 2022] PUT /identity/v3/projects/d61da770f51a492ea01d2aad14a35147/users/4061c8c2422a4fd68342f1fe6524f421/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:44.860257 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2392a44-d5b4-45fb-a13f-de45970d2c78 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:44.868241 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-a2392a44-d5b4-45fb-a13f-de45970d2c78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:44.875876 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-09332cc7-1fff-44ab-b984-6e264d4ef79a None None] MFA Rules not processed for user `2bd47bf9aa4142b1b7ec8d02e8661578`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:44.884107 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-09332cc7-1fff-44ab-b984-6e264d4ef79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:44.966515 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:45.068945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 896/1781] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:45.071446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2392a44-d5b4-45fb-a13f-de45970d2c78 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:45.078191 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f4250-2a34-4cd0-a5ba-abef2efdfadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:45.080993 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f4250-2a34-4cd0-a5ba-abef2efdfadf admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:45.081297 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09332cc7-1fff-44ab-b984-6e264d4ef79a None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:45.081469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f4250-2a34-4cd0-a5ba-abef2efdfadf admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:45.081998 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f4250-2a34-4cd0-a5ba-abef2efdfadf admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:45.082737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f4250-2a34-4cd0-a5ba-abef2efdfadf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:45.085109 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f4250-2a34-4cd0-a5ba-abef2efdfadf admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:45.090324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-09332cc7-1fff-44ab-b984-6e264d4ef79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:45.133815 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 886/1782] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:12:44 2022] POST /identity/v3/projects => generated 367 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:45.137408 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac876b-7629-4df3-bee4-2895112ef376 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:45.137835 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac876b-7629-4df3-bee4-2895112ef376 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:45.138569 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac876b-7629-4df3-bee4-2895112ef376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:45.189009 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-9fac876b-7629-4df3-bee4-2895112ef376 None None] MFA Rules not processed for user `4061c8c2422a4fd68342f1fe6524f421`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:45.201002 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-9fac876b-7629-4df3-bee4-2895112ef376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:45.243882 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09332cc7-1fff-44ab-b984-6e264d4ef79a None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:45.305628 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:45.406415 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 887/1783] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:45.406415 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac876b-7629-4df3-bee4-2895112ef376 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:45.440922 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456749f4-b0c2-451a-b364-b9e1335b97ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:45.443478 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456749f4-b0c2-451a-b364-b9e1335b97ce service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:45.443634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456749f4-b0c2-451a-b364-b9e1335b97ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:45.443862 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456749f4-b0c2-451a-b364-b9e1335b97ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:45.607661 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac876b-7629-4df3-bee4-2895112ef376 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:45.619235 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea6e2c-337c-46fc-8fa7-3c2946e23ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:45.620159 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea6e2c-337c-46fc-8fa7-3c2946e23ff9 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:45.621657 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea6e2c-337c-46fc-8fa7-3c2946e23ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:45.621943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea6e2c-337c-46fc-8fa7-3c2946e23ff9 admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:45.624839 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea6e2c-337c-46fc-8fa7-3c2946e23ff9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:45.632484 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea6e2c-337c-46fc-8fa7-3c2946e23ff9 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:45.681599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 888/1784] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:12:45 2022] POST /identity/v3/users => generated 290 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:45.681959 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4333bc1-0e66-4de0-8f50-3186c54fd744 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:45.682243 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4333bc1-0e66-4de0-8f50-3186c54fd744 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:45.682511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4333bc1-0e66-4de0-8f50-3186c54fd744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:45.740758 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456749f4-b0c2-451a-b364-b9e1335b97ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:45.744187 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456749f4-b0c2-451a-b364-b9e1335b97ce service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:45.786164 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 897/1785] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 717 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:45.788506 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456749f4-b0c2-451a-b364-b9e1335b97ce service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:45.824872 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456749f4-b0c2-451a-b364-b9e1335b97ce service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:45.830422 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-c4333bc1-0e66-4de0-8f50-3186c54fd744 None None] MFA Rules not processed for user `3fada78ceaeb4f3bad77c516f6dfcfd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:45.837559 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c4333bc1-0e66-4de0-8f50-3186c54fd744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:45.846384 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b55dc-64d5-49a3-a56f-9c4294d49115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:45.847421 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b55dc-64d5-49a3-a56f-9c4294d49115 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:45.847868 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b55dc-64d5-49a3-a56f-9c4294d49115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:45.848281 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b55dc-64d5-49a3-a56f-9c4294d49115 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:45.849870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b55dc-64d5-49a3-a56f-9c4294d49115 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:45.859402 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b55dc-64d5-49a3-a56f-9c4294d49115 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:45.978805 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 898/1786] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:12:45 2022] POST /identity/v3/projects => generated 385 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:45.986254 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b55dc-64d5-49a3-a56f-9c4294d49115 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:45.991858 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-632b55dc-64d5-49a3-a56f-9c4294d49115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:46.033658 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 889/1787] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 15:12:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:46.047086 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4333bc1-0e66-4de0-8f50-3186c54fd744 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:46.077101 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4333bc1-0e66-4de0-8f50-3186c54fd744 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:46.139752 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:46.151058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:46.215455 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c17a6-1260-4e7c-b269-ffa6f4330e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:46.216165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c17a6-1260-4e7c-b269-ffa6f4330e2c admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:46.216441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c17a6-1260-4e7c-b269-ffa6f4330e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:46.216735 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c17a6-1260-4e7c-b269-ffa6f4330e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:46.217597 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c17a6-1260-4e7c-b269-ffa6f4330e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:46.219896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c17a6-1260-4e7c-b269-ffa6f4330e2c admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:46.253252 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 890/1788] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:46.260142 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c17a6-1260-4e7c-b269-ffa6f4330e2c admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:46.266885 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b55dc-64d5-49a3-a56f-9c4294d49115 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:46.276647 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17532f86-1951-400f-94ef-46f5415f4f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:46.281533 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17532f86-1951-400f-94ef-46f5415f4f96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:46.282296 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17532f86-1951-400f-94ef-46f5415f4f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:46.282676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17532f86-1951-400f-94ef-46f5415f4f96 admin admin] PATH_INFO: `/v3/users/3c9aebb52950445d81c18ce551fe06b3` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:46.284605 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17532f86-1951-400f-94ef-46f5415f4f96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c9aebb52950445d81c18ce551fe06b3)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:46.287210 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17532f86-1951-400f-94ef-46f5415f4f96 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:46.310525 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c17a6-1260-4e7c-b269-ffa6f4330e2c admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:46.325662 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bccdf2-db6b-41a0-b648-9436f4b8900d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:46.327300 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bccdf2-db6b-41a0-b648-9436f4b8900d admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:46.327954 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bccdf2-db6b-41a0-b648-9436f4b8900d admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:46.328653 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bccdf2-db6b-41a0-b648-9436f4b8900d admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:46.330137 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bccdf2-db6b-41a0-b648-9436f4b8900d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:46.336344 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bccdf2-db6b-41a0-b648-9436f4b8900d admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:46.388897 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 891/1789] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:12:46 2022] POST /identity/v3/users => generated 299 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:46.394236 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bccdf2-db6b-41a0-b648-9436f4b8900d admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:46.419403 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bccdf2-db6b-41a0-b648-9436f4b8900d admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:46.420408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-17532f86-1951-400f-94ef-46f5415f4f96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c9aebb52950445d81c18ce551fe06b3'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:46.432312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f372d68-cac7-44a1-99f8-51141f41554b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:46.433055 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f372d68-cac7-44a1-99f8-51141f41554b admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:46.433367 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f372d68-cac7-44a1-99f8-51141f41554b admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:46.433692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f372d68-cac7-44a1-99f8-51141f41554b admin admin] PATH_INFO: `/v3/projects/a840430652fd435bbe83189ad842b751/users/78d03ccac11a452b8f3fdb455c659751/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:46.447001 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-17532f86-1951-400f-94ef-46f5415f4f96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c9aebb52950445d81c18ce551fe06b3'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:46.475731 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f372d68-cac7-44a1-99f8-51141f41554b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a840430652fd435bbe83189ad842b751, user_id=78d03ccac11a452b8f3fdb455c659751, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:46.478758 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f372d68-cac7-44a1-99f8-51141f41554b admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:46.485987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-17532f86-1951-400f-94ef-46f5415f4f96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c9aebb52950445d81c18ce551fe06b3'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:46.487011 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-17532f86-1951-400f-94ef-46f5415f4f96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c9aebb52950445d81c18ce551fe06b3'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:46.509588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 892/1790] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:46 2022] PUT /identity/v3/projects/a840430652fd435bbe83189ad842b751/users/78d03ccac11a452b8f3fdb455c659751/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:46.511996 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f372d68-cac7-44a1-99f8-51141f41554b admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:46.515462 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-0f372d68-cac7-44a1-99f8-51141f41554b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:46.523442 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-17532f86-1951-400f-94ef-46f5415f4f96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c9aebb52950445d81c18ce551fe06b3'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:46.540766 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 899/1791] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:12:45 2022] DELETE /identity/v3/users/3c9aebb52950445d81c18ce551fe06b3 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:46.542064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17532f86-1951-400f-94ef-46f5415f4f96 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:46.544862 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-17532f86-1951-400f-94ef-46f5415f4f96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:46.669268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17532f86-1951-400f-94ef-46f5415f4f96 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:46.682507 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7decae1-599c-479d-8a95-dbe051d68242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:46.683264 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7decae1-599c-479d-8a95-dbe051d68242 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:46.683640 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7decae1-599c-479d-8a95-dbe051d68242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:46.693212 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7decae1-599c-479d-8a95-dbe051d68242 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:46.694362 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7decae1-599c-479d-8a95-dbe051d68242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:46.697284 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7decae1-599c-479d-8a95-dbe051d68242 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:46.724620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 900/1792] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:46.724620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7decae1-599c-479d-8a95-dbe051d68242 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:46.726721 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-e7decae1-599c-479d-8a95-dbe051d68242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:46.741117 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7decae1-599c-479d-8a95-dbe051d68242 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:46.747092 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a0053-ed04-48cd-9f28-11783f955935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:46.747988 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a0053-ed04-48cd-9f28-11783f955935 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:46.748392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a0053-ed04-48cd-9f28-11783f955935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:46.748644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a0053-ed04-48cd-9f28-11783f955935 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:46.749403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a0053-ed04-48cd-9f28-11783f955935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:46.751143 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a0053-ed04-48cd-9f28-11783f955935 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:46.752411 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f372d68-cac7-44a1-99f8-51141f41554b admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:46.763223 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 901/1793] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:46.764715 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84464b08-c3c9-4b8c-931b-5422697ddb63 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:46.765064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84464b08-c3c9-4b8c-931b-5422697ddb63 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:46.765425 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84464b08-c3c9-4b8c-931b-5422697ddb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:46.792309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-84464b08-c3c9-4b8c-931b-5422697ddb63 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:46.793909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-84464b08-c3c9-4b8c-931b-5422697ddb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:46.804708 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 902/1794] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:46.805567 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84464b08-c3c9-4b8c-931b-5422697ddb63 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:46.832172 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84464b08-c3c9-4b8c-931b-5422697ddb63 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:46.837859 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1be39-5649-4c6c-9910-41fd5a85fbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:46.839084 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1be39-5649-4c6c-9910-41fd5a85fbd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:46.839551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1be39-5649-4c6c-9910-41fd5a85fbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:46.840403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1be39-5649-4c6c-9910-41fd5a85fbd5 admin admin] PATH_INFO: `/v3/projects/b6f10386e00b49e28c8bdf1ed9b9b710/users/ac8d8af2e45e45eab8180c39daca6535/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:46.860310 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:46.869353 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1be39-5649-4c6c-9910-41fd5a85fbd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6f10386e00b49e28c8bdf1ed9b9b710, user_id=ac8d8af2e45e45eab8180c39daca6535, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:46.877799 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1be39-5649-4c6c-9910-41fd5a85fbd5 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:46.902869 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c45f6-7133-4900-a182-fa32f8d65537 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05aa5e40ee4542719657bc03b2015e72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:46.904110 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c45f6-7133-4900-a182-fa32f8d65537 service swift] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:46.904728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c45f6-7133-4900-a182-fa32f8d65537 service swift] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:46.905442 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c45f6-7133-4900-a182-fa32f8d65537 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:46.909521 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-239c45f6-7133-4900-a182-fa32f8d65537 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:46.929986 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 903/1795] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:46 2022] PUT /identity/v3/projects/b6f10386e00b49e28c8bdf1ed9b9b710/users/ac8d8af2e45e45eab8180c39daca6535/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:46.929986 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1be39-5649-4c6c-9910-41fd5a85fbd5 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:46.932810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:47.061096 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c45f6-7133-4900-a182-fa32f8d65537 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:47.063279 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c45f6-7133-4900-a182-fa32f8d65537 service swift] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:47.129067 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 893/1796] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 15:12:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 15:12:47.130069 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c45f6-7133-4900-a182-fa32f8d65537 service swift] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:47.137031 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-239c45f6-7133-4900-a182-fa32f8d65537 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:47.198205 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1be39-5649-4c6c-9910-41fd5a85fbd5 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:47.218169 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e977b3-dddc-4bf5-89bc-a04e9e90930a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:47.223293 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e977b3-dddc-4bf5-89bc-a04e9e90930a admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:47.227521 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e977b3-dddc-4bf5-89bc-a04e9e90930a admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:47.227893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e977b3-dddc-4bf5-89bc-a04e9e90930a admin admin] PATH_INFO: `/v3/projects/a840430652fd435bbe83189ad842b751/users/78d03ccac11a452b8f3fdb455c659751/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:47.236699 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e977b3-dddc-4bf5-89bc-a04e9e90930a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a840430652fd435bbe83189ad842b751, user_id=78d03ccac11a452b8f3fdb455c659751, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:47.243978 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e977b3-dddc-4bf5-89bc-a04e9e90930a admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:47.260013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 904/1797] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:46 2022] PUT /identity/v3/projects/a840430652fd435bbe83189ad842b751/users/78d03ccac11a452b8f3fdb455c659751/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:47.261275 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e977b3-dddc-4bf5-89bc-a04e9e90930a admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:47.441537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c45f6-7133-4900-a182-fa32f8d65537 service swift] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:47.504870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e977b3-dddc-4bf5-89bc-a04e9e90930a admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:47.512781 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee3ca8-9882-4d83-9eb0-98059baa4cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:47.513431 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee3ca8-9882-4d83-9eb0-98059baa4cae admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:47.513936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee3ca8-9882-4d83-9eb0-98059baa4cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:47.514210 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee3ca8-9882-4d83-9eb0-98059baa4cae admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:47.515077 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fee3ca8-9882-4d83-9eb0-98059baa4cae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:47.518290 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fee3ca8-9882-4d83-9eb0-98059baa4cae admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:47.544957 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 905/1798] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:47.549169 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc231440-9625-44d2-b07c-d70fb675dffb None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:47.549992 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc231440-9625-44d2-b07c-d70fb675dffb None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:47.550254 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc231440-9625-44d2-b07c-d70fb675dffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:47.606248 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-dc231440-9625-44d2-b07c-d70fb675dffb None None] MFA Rules not processed for user `697b73f182e6419a84e0c8c6b0fc0c06`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:47.607687 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-dc231440-9625-44d2-b07c-d70fb675dffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:47.623587 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2a751-e46c-435a-9dfc-bad4d1092397 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:47.626095 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2a751-e46c-435a-9dfc-bad4d1092397 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:47.626544 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2a751-e46c-435a-9dfc-bad4d1092397 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:47.626970 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2a751-e46c-435a-9dfc-bad4d1092397 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:47.636235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 906/1799] 10.222.0.46 () {58 vars in 1015 bytes} [Sun Sep 11 15:12:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:47.640632 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c5455-4c34-4f9b-9e64-ee2c7d3e6d7a None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:47.641365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c5455-4c34-4f9b-9e64-ee2c7d3e6d7a None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:47.642130 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c5455-4c34-4f9b-9e64-ee2c7d3e6d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:47.652407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e2a751-e46c-435a-9dfc-bad4d1092397 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:47.656586 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e2a751-e46c-435a-9dfc-bad4d1092397 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:47.703463 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 894/1800] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:47.704657 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2a751-e46c-435a-9dfc-bad4d1092397 service neutron] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:47.725967 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-7a2c5455-4c34-4f9b-9e64-ee2c7d3e6d7a None None] MFA Rules not processed for user `78d03ccac11a452b8f3fdb455c659751`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:47.730997 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-7a2c5455-4c34-4f9b-9e64-ee2c7d3e6d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:47.731446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2a751-e46c-435a-9dfc-bad4d1092397 service neutron] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:47.743132 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c84959-b7d6-41ce-932e-bae90b46416d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:47.747123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c84959-b7d6-41ce-932e-bae90b46416d admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:47.747665 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c84959-b7d6-41ce-932e-bae90b46416d admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:47.748076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c84959-b7d6-41ce-932e-bae90b46416d admin admin] PATH_INFO: `/v3/projects/b6f10386e00b49e28c8bdf1ed9b9b710/users/ac8d8af2e45e45eab8180c39daca6535/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:47.753195 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c84959-b7d6-41ce-932e-bae90b46416d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6f10386e00b49e28c8bdf1ed9b9b710, user_id=ac8d8af2e45e45eab8180c39daca6535, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:47.761707 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c84959-b7d6-41ce-932e-bae90b46416d admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:47.805073 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 895/1801] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:47 2022] PUT /identity/v3/projects/b6f10386e00b49e28c8bdf1ed9b9b710/users/ac8d8af2e45e45eab8180c39daca6535/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:47.809934 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636861bd-65ab-4536-9359-4f6c6b2cacaf None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:47.824316 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636861bd-65ab-4536-9359-4f6c6b2cacaf None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:47.827180 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636861bd-65ab-4536-9359-4f6c6b2cacaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:47.955430 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:47.956779 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-636861bd-65ab-4536-9359-4f6c6b2cacaf None None] MFA Rules not processed for user `ac8d8af2e45e45eab8180c39daca6535`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:47.962936 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-636861bd-65ab-4536-9359-4f6c6b2cacaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:47.965700 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 907/1802] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:47 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:47.967898 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c5455-4c34-4f9b-9e64-ee2c7d3e6d7a None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:48.011303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c5455-4c34-4f9b-9e64-ee2c7d3e6d7a None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:48.260943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:48.270301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5dc74-cef7-4f37-960b-c8f8fd5e5c21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:48.272473 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5dc74-cef7-4f37-960b-c8f8fd5e5c21 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:48.274179 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5dc74-cef7-4f37-960b-c8f8fd5e5c21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:48.276917 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5dc74-cef7-4f37-960b-c8f8fd5e5c21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:48.310412 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5dc74-cef7-4f37-960b-c8f8fd5e5c21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:48.314699 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5dc74-cef7-4f37-960b-c8f8fd5e5c21 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:48.355206 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 908/1803] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:48.357965 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5dc74-cef7-4f37-960b-c8f8fd5e5c21 service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:48.360341 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-89c5dc74-cef7-4f37-960b-c8f8fd5e5c21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:48.407993 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 896/1804] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:47 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 602 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:48.408888 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636861bd-65ab-4536-9359-4f6c6b2cacaf None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:48.430897 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-636861bd-65ab-4536-9359-4f6c6b2cacaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:48.531762 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5dc74-cef7-4f37-960b-c8f8fd5e5c21 service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:48.544535 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad04e2-6068-4daf-a838-f50471738304 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:48.545207 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad04e2-6068-4daf-a838-f50471738304 service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:48.545510 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad04e2-6068-4daf-a838-f50471738304 service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:48.545762 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad04e2-6068-4daf-a838-f50471738304 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:48.784607 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636861bd-65ab-4536-9359-4f6c6b2cacaf None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:48.800107 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad04e2-6068-4daf-a838-f50471738304 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:48.801596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad04e2-6068-4daf-a838-f50471738304 service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:48.811082 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d33967-5722-4c9a-b551-ecd6cb6d05e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:48.811727 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d33967-5722-4c9a-b551-ecd6cb6d05e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:48.812005 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d33967-5722-4c9a-b551-ecd6cb6d05e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:48.812278 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d33967-5722-4c9a-b551-ecd6cb6d05e0 admin admin] PATH_INFO: `/v3/projects/01d166e17c174e74ac0f1203c36c12c7` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:48.819117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d33967-5722-4c9a-b551-ecd6cb6d05e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01d166e17c174e74ac0f1203c36c12c7)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:48.827255 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d33967-5722-4c9a-b551-ecd6cb6d05e0 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:48.837925 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 909/1805] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:48 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:48.838934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad04e2-6068-4daf-a838-f50471738304 service cinder] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:48.847371 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-05ad04e2-6068-4daf-a838-f50471738304 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:48.890057 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad04e2-6068-4daf-a838-f50471738304 service cinder] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:48.901242 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aabde9-6174-40e1-b693-750e370c10c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:48.904901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aabde9-6174-40e1-b693-750e370c10c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:48.907431 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aabde9-6174-40e1-b693-750e370c10c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:48.909127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aabde9-6174-40e1-b693-750e370c10c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:48.932502 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aabde9-6174-40e1-b693-750e370c10c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:48.937291 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aabde9-6174-40e1-b693-750e370c10c0 service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:48.982782 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 910/1806] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:48 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:49.086091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-03d33967-5722-4c9a-b551-ecd6cb6d05e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01d166e17c174e74ac0f1203c36c12c7', 'request_id': 'req-03d33967-5722-4c9a-b551-ecd6cb6d05e0'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:49.113012 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-03d33967-5722-4c9a-b551-ecd6cb6d05e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01d166e17c174e74ac0f1203c36c12c7', 'request_id': 'req-03d33967-5722-4c9a-b551-ecd6cb6d05e0'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:49.134302 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-03d33967-5722-4c9a-b551-ecd6cb6d05e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01d166e17c174e74ac0f1203c36c12c7', 'request_id': 'req-03d33967-5722-4c9a-b551-ecd6cb6d05e0'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:49.151324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-03d33967-5722-4c9a-b551-ecd6cb6d05e0 admin admin] The token cache is being invalidate because project 01d166e17c174e74ac0f1203c36c12c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:12:49.151846 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-03d33967-5722-4c9a-b551-ecd6cb6d05e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:49.153108 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-03d33967-5722-4c9a-b551-ecd6cb6d05e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:49.158857 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 897/1807] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:12:48 2022] DELETE /identity/v3/projects/01d166e17c174e74ac0f1203c36c12c7 => generated 0 bytes in 747 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:49.160087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aabde9-6174-40e1-b693-750e370c10c0 service cinder] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:49.187740 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aabde9-6174-40e1-b693-750e370c10c0 service cinder] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:49.398957 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:49.486834 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d2c672-ea93-4dc4-876b-a6a8cdf93dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:49.487518 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2c672-ea93-4dc4-876b-a6a8cdf93dbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:49.487769 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2c672-ea93-4dc4-876b-a6a8cdf93dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:49.488039 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2c672-ea93-4dc4-876b-a6a8cdf93dbb admin admin] PATH_INFO: `/v3/users/9635f4ef205146e5a943de36ee03377a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:49.497296 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d2c672-ea93-4dc4-876b-a6a8cdf93dbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9635f4ef205146e5a943de36ee03377a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:49.508769 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d2c672-ea93-4dc4-876b-a6a8cdf93dbb admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:49.691315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-c6d2c672-ea93-4dc4-876b-a6a8cdf93dbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9635f4ef205146e5a943de36ee03377a'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:49.717358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-c6d2c672-ea93-4dc4-876b-a6a8cdf93dbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9635f4ef205146e5a943de36ee03377a'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:49.804938 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-c6d2c672-ea93-4dc4-876b-a6a8cdf93dbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9635f4ef205146e5a943de36ee03377a'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:49.808046 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-c6d2c672-ea93-4dc4-876b-a6a8cdf93dbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9635f4ef205146e5a943de36ee03377a'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:49.865747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-c6d2c672-ea93-4dc4-876b-a6a8cdf93dbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9635f4ef205146e5a943de36ee03377a'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:49.936760 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 911/1808] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:12:49 2022] DELETE /identity/v3/users/9635f4ef205146e5a943de36ee03377a => generated 0 bytes in 774 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:50.624652 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d33967-5722-4c9a-b551-ecd6cb6d05e0 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:50.626502 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-03d33967-5722-4c9a-b551-ecd6cb6d05e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:50.758881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d2c672-ea93-4dc4-876b-a6a8cdf93dbb admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:50.760437 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-c6d2c672-ea93-4dc4-876b-a6a8cdf93dbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:50.810359 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d33967-5722-4c9a-b551-ecd6cb6d05e0 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:50.951293 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:50.990122 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d2c672-ea93-4dc4-876b-a6a8cdf93dbb admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:50.995436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05439e1e-f872-4623-adf0-85e7a522f7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:50.996568 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05439e1e-f872-4623-adf0-85e7a522f7ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:50.997050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05439e1e-f872-4623-adf0-85e7a522f7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:50.997507 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05439e1e-f872-4623-adf0-85e7a522f7ca admin admin] PATH_INFO: `/v3/projects/80b9bbfd704c48c292a345ba08af93b9` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:50.997631 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2f13b-09ec-487a-9fa7-d40a69ffb293 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:51.000250 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2f13b-09ec-487a-9fa7-d40a69ffb293 service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:51.000530 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2f13b-09ec-487a-9fa7-d40a69ffb293 service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:51.001114 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2f13b-09ec-487a-9fa7-d40a69ffb293 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:51.007536 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05439e1e-f872-4623-adf0-85e7a522f7ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=80b9bbfd704c48c292a345ba08af93b9)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:51.007974 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-63e2f13b-09ec-487a-9fa7-d40a69ffb293 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:51.010091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05439e1e-f872-4623-adf0-85e7a522f7ca admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:51.095053 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-05439e1e-f872-4623-adf0-85e7a522f7ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80b9bbfd704c48c292a345ba08af93b9', 'request_id': 'req-05439e1e-f872-4623-adf0-85e7a522f7ca'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:51.112311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-05439e1e-f872-4623-adf0-85e7a522f7ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80b9bbfd704c48c292a345ba08af93b9', 'request_id': 'req-05439e1e-f872-4623-adf0-85e7a522f7ca'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:51.115283 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:51.122047 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-05439e1e-f872-4623-adf0-85e7a522f7ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80b9bbfd704c48c292a345ba08af93b9', 'request_id': 'req-05439e1e-f872-4623-adf0-85e7a522f7ca'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:51.143520 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-05439e1e-f872-4623-adf0-85e7a522f7ca admin admin] The token cache is being invalidate because project 80b9bbfd704c48c292a345ba08af93b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:12:51.143879 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-05439e1e-f872-4623-adf0-85e7a522f7ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:51.144698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-05439e1e-f872-4623-adf0-85e7a522f7ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:51.147664 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 912/1809] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:12:50 2022] DELETE /identity/v3/projects/80b9bbfd704c48c292a345ba08af93b9 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:12:51.148686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2f13b-09ec-487a-9fa7-d40a69ffb293 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:51.151311 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2f13b-09ec-487a-9fa7-d40a69ffb293 service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:51.152989 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05439e1e-f872-4623-adf0-85e7a522f7ca admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:51.155867 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-05439e1e-f872-4623-adf0-85e7a522f7ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:51.189567 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 898/1810] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:50 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:51.347884 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05439e1e-f872-4623-adf0-85e7a522f7ca admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:51.362566 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd603e-f49a-4745-a670-8e448422ab37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:51.363629 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd603e-f49a-4745-a670-8e448422ab37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:51.364166 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd603e-f49a-4745-a670-8e448422ab37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:51.364692 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd603e-f49a-4745-a670-8e448422ab37 admin admin] PATH_INFO: `/v3/users/3cf7193cea0344169df1b5d549f02a83` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:51.368038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd603e-f49a-4745-a670-8e448422ab37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cf7193cea0344169df1b5d549f02a83)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:51.382468 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd603e-f49a-4745-a670-8e448422ab37 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:51.572148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e3bd603e-f49a-4745-a670-8e448422ab37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cf7193cea0344169df1b5d549f02a83'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:51.588320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e3bd603e-f49a-4745-a670-8e448422ab37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cf7193cea0344169df1b5d549f02a83'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:51.638763 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e3bd603e-f49a-4745-a670-8e448422ab37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cf7193cea0344169df1b5d549f02a83'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:51.639436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e3bd603e-f49a-4745-a670-8e448422ab37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cf7193cea0344169df1b5d549f02a83'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:51.652576 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e3bd603e-f49a-4745-a670-8e448422ab37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cf7193cea0344169df1b5d549f02a83'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:51.672629 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 913/1811] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:12:51 2022] DELETE /identity/v3/users/3cf7193cea0344169df1b5d549f02a83 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:52.263507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2f13b-09ec-487a-9fa7-d40a69ffb293 service cinder] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:52.266422 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-63e2f13b-09ec-487a-9fa7-d40a69ffb293 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:52.306740 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2f13b-09ec-487a-9fa7-d40a69ffb293 service cinder] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:52.483514 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a85a8-cd99-4b53-ab94-4345785a68f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:52.485001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a85a8-cd99-4b53-ab94-4345785a68f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:52.485536 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a85a8-cd99-4b53-ab94-4345785a68f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:52.486075 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a85a8-cd99-4b53-ab94-4345785a68f0 admin admin] PATH_INFO: `/v3/projects/163c5689a40e4f76841cb11e9f68bca5` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:52.487531 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a85a8-cd99-4b53-ab94-4345785a68f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=163c5689a40e4f76841cb11e9f68bca5)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:52.493239 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a85a8-cd99-4b53-ab94-4345785a68f0 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:52.596091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8a2a85a8-cd99-4b53-ab94-4345785a68f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '163c5689a40e4f76841cb11e9f68bca5', 'request_id': 'req-8a2a85a8-cd99-4b53-ab94-4345785a68f0'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:52.620043 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8a2a85a8-cd99-4b53-ab94-4345785a68f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '163c5689a40e4f76841cb11e9f68bca5', 'request_id': 'req-8a2a85a8-cd99-4b53-ab94-4345785a68f0'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:52.630304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8a2a85a8-cd99-4b53-ab94-4345785a68f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '163c5689a40e4f76841cb11e9f68bca5', 'request_id': 'req-8a2a85a8-cd99-4b53-ab94-4345785a68f0'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:52.653413 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a88549-44ff-40c6-a2ca-2ea211e2f4ae None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:52.653864 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a88549-44ff-40c6-a2ca-2ea211e2f4ae None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:52.654002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a88549-44ff-40c6-a2ca-2ea211e2f4ae None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:52.655938 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 914/1812] 10.222.0.44 () {54 vars in 921 bytes} [Sun Sep 11 15:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:12:52.667428 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8a2a85a8-cd99-4b53-ab94-4345785a68f0 admin admin] The token cache is being invalidate because project 163c5689a40e4f76841cb11e9f68bca5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:12:52.667632 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8a2a85a8-cd99-4b53-ab94-4345785a68f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:52.670520 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8a2a85a8-cd99-4b53-ab94-4345785a68f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:52.675865 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 899/1813] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:12:52 2022] DELETE /identity/v3/projects/163c5689a40e4f76841cb11e9f68bca5 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:52.681884 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50c695-d358-4c17-ae61-ee008aa3aedc None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:52.682357 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50c695-d358-4c17-ae61-ee008aa3aedc None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:52.682839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50c695-d358-4c17-ae61-ee008aa3aedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:52.706901 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a928bc-0dc5-40f7-b2c1-d308edfd9beb None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:52.707201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a928bc-0dc5-40f7-b2c1-d308edfd9beb None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:52.707494 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a928bc-0dc5-40f7-b2c1-d308edfd9beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:52.720273 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-3d50c695-d358-4c17-ae61-ee008aa3aedc None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:52.726444 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-3d50c695-d358-4c17-ae61-ee008aa3aedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:52.759104 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-14a928bc-0dc5-40f7-b2c1-d308edfd9beb None None] MFA Rules not processed for user `453297dac64046d19fc1c331d2d85abc`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:52.764767 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-14a928bc-0dc5-40f7-b2c1-d308edfd9beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:52.788363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:52.872393 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 915/1814] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:52.881772 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65850165-6f89-4283-88d8-d760d22f036d None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:52.882285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65850165-6f89-4283-88d8-d760d22f036d None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:52.882654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65850165-6f89-4283-88d8-d760d22f036d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:52.916738 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-65850165-6f89-4283-88d8-d760d22f036d None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:52.918175 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-65850165-6f89-4283-88d8-d760d22f036d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:52.936231 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 916/1815] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:52.937255 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65850165-6f89-4283-88d8-d760d22f036d None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:52.965403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:52.972352 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65850165-6f89-4283-88d8-d760d22f036d None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:52.988181 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b9fff-1e2d-4c1c-b4d7-9840d23a82bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:52.988859 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b9fff-1e2d-4c1c-b4d7-9840d23a82bd admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:52.989174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b9fff-1e2d-4c1c-b4d7-9840d23a82bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:52.989487 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b9fff-1e2d-4c1c-b4d7-9840d23a82bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:52.990344 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b9fff-1e2d-4c1c-b4d7-9840d23a82bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:12:52.990646 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b9fff-1e2d-4c1c-b4d7-9840d23a82bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:52.995345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b9fff-1e2d-4c1c-b4d7-9840d23a82bd admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:53.119343 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 917/1816] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:12:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:12:53.120316 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b9fff-1e2d-4c1c-b4d7-9840d23a82bd admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:53.144720 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 900/1817] 10.222.0.44 () {58 vars in 1018 bytes} [Sun Sep 11 15:12:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 439 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:53.149752 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a928bc-0dc5-40f7-b2c1-d308edfd9beb None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:53.156967 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-14a928bc-0dc5-40f7-b2c1-d308edfd9beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:53.191512 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b9fff-1e2d-4c1c-b4d7-9840d23a82bd admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:53.208850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a928bc-0dc5-40f7-b2c1-d308edfd9beb None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:53.210585 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9357d-7745-49a5-851b-d54173056e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:53.212058 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9357d-7745-49a5-851b-d54173056e7f admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:53.212918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9357d-7745-49a5-851b-d54173056e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:53.213473 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9357d-7745-49a5-851b-d54173056e7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:53.214830 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d9357d-7745-49a5-851b-d54173056e7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:53.218382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d9357d-7745-49a5-851b-d54173056e7f admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:53.224481 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5528f8-8e33-450d-b110-08586dba02bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:53.225162 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5528f8-8e33-450d-b110-08586dba02bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:53.229337 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5528f8-8e33-450d-b110-08586dba02bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:53.230342 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5528f8-8e33-450d-b110-08586dba02bd admin admin] PATH_INFO: `/v3/users/736f2eb232fe44f6a0a6cc9824a76e98` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:53.234923 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5528f8-8e33-450d-b110-08586dba02bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=736f2eb232fe44f6a0a6cc9824a76e98)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:53.242359 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5528f8-8e33-450d-b110-08586dba02bd admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:53.349874 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 918/1818] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:12:53 2022] POST /identity/v3/projects => generated 363 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:53.353762 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9357d-7745-49a5-851b-d54173056e7f admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:53.359934 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cd5528f8-8e33-450d-b110-08586dba02bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '736f2eb232fe44f6a0a6cc9824a76e98'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:53.377088 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cd5528f8-8e33-450d-b110-08586dba02bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '736f2eb232fe44f6a0a6cc9824a76e98'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:53.412062 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cd5528f8-8e33-450d-b110-08586dba02bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '736f2eb232fe44f6a0a6cc9824a76e98'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:53.413862 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cd5528f8-8e33-450d-b110-08586dba02bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '736f2eb232fe44f6a0a6cc9824a76e98'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:53.455168 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cd5528f8-8e33-450d-b110-08586dba02bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '736f2eb232fe44f6a0a6cc9824a76e98'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:53.490149 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 901/1819] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:12:53 2022] DELETE /identity/v3/users/736f2eb232fe44f6a0a6cc9824a76e98 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:53.498047 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebbc5f-dc2c-4a8e-977f-ece20ccc64a8 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:53.502111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebbc5f-dc2c-4a8e-977f-ece20ccc64a8 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:53.504097 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebbc5f-dc2c-4a8e-977f-ece20ccc64a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:53.576949 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9357d-7745-49a5-851b-d54173056e7f admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:53.604912 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-51ebbc5f-dc2c-4a8e-977f-ece20ccc64a8 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:53.606418 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-51ebbc5f-dc2c-4a8e-977f-ece20ccc64a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:53.743846 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 902/1820] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:53.747707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5de378-6257-4686-afd2-8c2490ed0411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:53.747707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5de378-6257-4686-afd2-8c2490ed0411 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:53.747707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5de378-6257-4686-afd2-8c2490ed0411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:53.747707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5de378-6257-4686-afd2-8c2490ed0411 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:53.748449 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5de378-6257-4686-afd2-8c2490ed0411 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:53.750666 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5de378-6257-4686-afd2-8c2490ed0411 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:53.751585 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebbc5f-dc2c-4a8e-977f-ece20ccc64a8 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:53.757560 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-51ebbc5f-dc2c-4a8e-977f-ece20ccc64a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:53.814266 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 919/1821] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:12:53 2022] POST /identity/v3/users => generated 282 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:53.818952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5de378-6257-4686-afd2-8c2490ed0411 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:53.826660 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-0f5de378-6257-4686-afd2-8c2490ed0411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:53.835284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:53.851836 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5de378-6257-4686-afd2-8c2490ed0411 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:53.865974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42df14-f30d-4bda-b39e-cb90a1d8e720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:53.866825 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42df14-f30d-4bda-b39e-cb90a1d8e720 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:53.868410 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42df14-f30d-4bda-b39e-cb90a1d8e720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:53.872206 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42df14-f30d-4bda-b39e-cb90a1d8e720 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:53.873170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f42df14-f30d-4bda-b39e-cb90a1d8e720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:53.879870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f42df14-f30d-4bda-b39e-cb90a1d8e720 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:53.915208 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 920/1822] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:53 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:53.916349 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42df14-f30d-4bda-b39e-cb90a1d8e720 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:53.923184 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-3f42df14-f30d-4bda-b39e-cb90a1d8e720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:54.022974 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebbc5f-dc2c-4a8e-977f-ece20ccc64a8 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:54.043132 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8fe39c-74e7-474f-83cb-7bb2bd4c9896 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:54.043914 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8fe39c-74e7-474f-83cb-7bb2bd4c9896 service nova] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:54.045823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8fe39c-74e7-474f-83cb-7bb2bd4c9896 service nova] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:54.046196 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8fe39c-74e7-474f-83cb-7bb2bd4c9896 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:54.052924 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-7b8fe39c-74e7-474f-83cb-7bb2bd4c9896 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:54.090927 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8fe39c-74e7-474f-83cb-7bb2bd4c9896 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:54.098472 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8fe39c-74e7-474f-83cb-7bb2bd4c9896 service nova] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:54.124518 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42df14-f30d-4bda-b39e-cb90a1d8e720 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:54.141852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f364cd-f4f9-4ea3-9f75-b3f678ddb67b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:54.142667 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f364cd-f4f9-4ea3-9f75-b3f678ddb67b service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:54.143026 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f364cd-f4f9-4ea3-9f75-b3f678ddb67b service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:54.144453 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f364cd-f4f9-4ea3-9f75-b3f678ddb67b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:54.165722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:54.170108 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f364cd-f4f9-4ea3-9f75-b3f678ddb67b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:54.173332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f364cd-f4f9-4ea3-9f75-b3f678ddb67b service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:54.179357 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 903/1823] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:54.180289 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8fe39c-74e7-474f-83cb-7bb2bd4c9896 service nova] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:54.194609 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 921/1824] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:54.268899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8fe39c-74e7-474f-83cb-7bb2bd4c9896 service nova] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:54.300177 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b106838-2c2d-4fbb-b17e-83a51fa08961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:54.300988 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b106838-2c2d-4fbb-b17e-83a51fa08961 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:54.301297 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b106838-2c2d-4fbb-b17e-83a51fa08961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:54.301566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b106838-2c2d-4fbb-b17e-83a51fa08961 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:54.302227 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b106838-2c2d-4fbb-b17e-83a51fa08961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:54.305451 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b106838-2c2d-4fbb-b17e-83a51fa08961 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:54.340520 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 904/1825] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:54.341453 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f364cd-f4f9-4ea3-9f75-b3f678ddb67b service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:54.356489 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f364cd-f4f9-4ea3-9f75-b3f678ddb67b service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:54.379780 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaed1e5-8fd5-4ce9-978f-949ef187aa23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:54.381550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaed1e5-8fd5-4ce9-978f-949ef187aa23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:54.381849 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaed1e5-8fd5-4ce9-978f-949ef187aa23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:54.382196 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaed1e5-8fd5-4ce9-978f-949ef187aa23 admin admin] PATH_INFO: `/v3/projects/5b1e9b6b29bd4dc49b05acd1764e0a40/users/0d326898402547a5beca3aaf9b5c2c62/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:54.427392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaed1e5-8fd5-4ce9-978f-949ef187aa23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b1e9b6b29bd4dc49b05acd1764e0a40, user_id=0d326898402547a5beca3aaf9b5c2c62, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:54.430617 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaed1e5-8fd5-4ce9-978f-949ef187aa23 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:54.480267 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 922/1826] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:54 2022] PUT /identity/v3/projects/5b1e9b6b29bd4dc49b05acd1764e0a40/users/0d326898402547a5beca3aaf9b5c2c62/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:54.484374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a470961-dde9-4b1e-bb89-7f19cc685072 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:54.485284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a470961-dde9-4b1e-bb89-7f19cc685072 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:54.485467 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a470961-dde9-4b1e-bb89-7f19cc685072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:54.556337 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-7a470961-dde9-4b1e-bb89-7f19cc685072 None None] MFA Rules not processed for user `0d326898402547a5beca3aaf9b5c2c62`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:54.561630 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-7a470961-dde9-4b1e-bb89-7f19cc685072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:54.672802 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaed1e5-8fd5-4ce9-978f-949ef187aa23 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:54.674967 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-dcaed1e5-8fd5-4ce9-978f-949ef187aa23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:54.763911 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 905/1827] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:54.765079 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a470961-dde9-4b1e-bb89-7f19cc685072 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:54.828771 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaed1e5-8fd5-4ce9-978f-949ef187aa23 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:54.848042 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e91c53-2eea-4da6-bdef-e04e5f07ab74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:54.848403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e91c53-2eea-4da6-bdef-e04e5f07ab74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:54.848685 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e91c53-2eea-4da6-bdef-e04e5f07ab74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:54.848955 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e91c53-2eea-4da6-bdef-e04e5f07ab74 admin admin] PATH_INFO: `/v3/projects/bdcdd9681fe344c886f6afbf5b93bf12` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:54.855440 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e91c53-2eea-4da6-bdef-e04e5f07ab74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdcdd9681fe344c886f6afbf5b93bf12)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:54.857403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e91c53-2eea-4da6-bdef-e04e5f07ab74 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:54.929795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a470961-dde9-4b1e-bb89-7f19cc685072 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:54.942964 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf2289-c5fe-42fc-8e54-95fc5341daa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:54.944395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf2289-c5fe-42fc-8e54-95fc5341daa7 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:54.944395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf2289-c5fe-42fc-8e54-95fc5341daa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:54.944395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf2289-c5fe-42fc-8e54-95fc5341daa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:54.944918 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebf2289-c5fe-42fc-8e54-95fc5341daa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:54.950444 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebf2289-c5fe-42fc-8e54-95fc5341daa7 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:55.014188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-72e91c53-2eea-4da6-bdef-e04e5f07ab74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdcdd9681fe344c886f6afbf5b93bf12', 'request_id': 'req-72e91c53-2eea-4da6-bdef-e04e5f07ab74'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:55.017441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 906/1828] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:12:54 2022] POST /identity/v3/projects => generated 365 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:55.020272 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf2289-c5fe-42fc-8e54-95fc5341daa7 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:55.036517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-72e91c53-2eea-4da6-bdef-e04e5f07ab74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdcdd9681fe344c886f6afbf5b93bf12', 'request_id': 'req-72e91c53-2eea-4da6-bdef-e04e5f07ab74'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:55.044418 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:55.057310 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-72e91c53-2eea-4da6-bdef-e04e5f07ab74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdcdd9681fe344c886f6afbf5b93bf12', 'request_id': 'req-72e91c53-2eea-4da6-bdef-e04e5f07ab74'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:55.097843 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-72e91c53-2eea-4da6-bdef-e04e5f07ab74 admin admin] The token cache is being invalidate because project bdcdd9681fe344c886f6afbf5b93bf12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:12:55.098136 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-72e91c53-2eea-4da6-bdef-e04e5f07ab74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:55.100025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-72e91c53-2eea-4da6-bdef-e04e5f07ab74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:55.103343 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 923/1829] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:12:54 2022] DELETE /identity/v3/projects/bdcdd9681fe344c886f6afbf5b93bf12 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:12:55.108271 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33fcec-772e-4bcc-a0c2-168151bbcdba None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:55.108552 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33fcec-772e-4bcc-a0c2-168151bbcdba None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:55.109003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33fcec-772e-4bcc-a0c2-168151bbcdba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:55.138325 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-3f33fcec-772e-4bcc-a0c2-168151bbcdba None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:55.140011 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-3f33fcec-772e-4bcc-a0c2-168151bbcdba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:55.167576 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf2289-c5fe-42fc-8e54-95fc5341daa7 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:55.173564 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8573a134-f47c-4d6f-b23e-54542f3cb415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:55.174257 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573a134-f47c-4d6f-b23e-54542f3cb415 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:55.174733 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573a134-f47c-4d6f-b23e-54542f3cb415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:55.174996 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573a134-f47c-4d6f-b23e-54542f3cb415 admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:55.175795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8573a134-f47c-4d6f-b23e-54542f3cb415 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:55.178017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8573a134-f47c-4d6f-b23e-54542f3cb415 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:55.204737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 924/1830] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:55.206432 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d542a1-eebc-46b5-813f-4b45bacdfec0 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:55.206710 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d542a1-eebc-46b5-813f-4b45bacdfec0 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:55.206982 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d542a1-eebc-46b5-813f-4b45bacdfec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:55.245071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 907/1831] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:12:55 2022] POST /identity/v3/users => generated 289 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:55.249846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-92d542a1-eebc-46b5-813f-4b45bacdfec0 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:55.250352 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8573a134-f47c-4d6f-b23e-54542f3cb415 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:55.251171 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-92d542a1-eebc-46b5-813f-4b45bacdfec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:55.256655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-8573a134-f47c-4d6f-b23e-54542f3cb415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:55.270586 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 925/1832] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:12:55.276351 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d542a1-eebc-46b5-813f-4b45bacdfec0 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:55.289168 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d542a1-eebc-46b5-813f-4b45bacdfec0 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:55.296624 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8573a134-f47c-4d6f-b23e-54542f3cb415 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:55.308219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0a3ae-790b-410a-863b-a8c6a65ee0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:55.309215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0a3ae-790b-410a-863b-a8c6a65ee0cc admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:55.309501 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0a3ae-790b-410a-863b-a8c6a65ee0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:55.309777 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0a3ae-790b-410a-863b-a8c6a65ee0cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:55.310558 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d0a3ae-790b-410a-863b-a8c6a65ee0cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:55.313030 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d0a3ae-790b-410a-863b-a8c6a65ee0cc admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:55.314842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188fb2d0-df23-4596-ba47-26b76381e2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:55.315474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fb2d0-df23-4596-ba47-26b76381e2a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:55.315749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fb2d0-df23-4596-ba47-26b76381e2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:55.316019 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fb2d0-df23-4596-ba47-26b76381e2a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:55.316681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188fb2d0-df23-4596-ba47-26b76381e2a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:12:55.316980 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188fb2d0-df23-4596-ba47-26b76381e2a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:55.320875 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188fb2d0-df23-4596-ba47-26b76381e2a2 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:55.329195 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 908/1833] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:12:55.334303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0a3ae-790b-410a-863b-a8c6a65ee0cc admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:55.372808 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 926/1834] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:12:55.375479 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188fb2d0-df23-4596-ba47-26b76381e2a2 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:55.385558 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0a3ae-790b-410a-863b-a8c6a65ee0cc admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:55.403030 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f0d52-ca50-42b9-af64-738add614fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:55.406487 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f0d52-ca50-42b9-af64-738add614fbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:55.406947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f0d52-ca50-42b9-af64-738add614fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:55.408130 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188fb2d0-df23-4596-ba47-26b76381e2a2 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:55.408710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f0d52-ca50-42b9-af64-738add614fbd admin admin] PATH_INFO: `/v3/projects/7fa5862a888d42fc8f85249f36c1a403/users/b872ce80cf3147ea8f31d63a137b3250/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:55.425487 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b55d6b-36b1-4694-a3b9-9ab9742909c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:55.429503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b55d6b-36b1-4694-a3b9-9ab9742909c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:55.432820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b55d6b-36b1-4694-a3b9-9ab9742909c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:55.433703 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b55d6b-36b1-4694-a3b9-9ab9742909c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:55.436520 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b55d6b-36b1-4694-a3b9-9ab9742909c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:55.441164 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b55d6b-36b1-4694-a3b9-9ab9742909c0 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:55.465884 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f0d52-ca50-42b9-af64-738add614fbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fa5862a888d42fc8f85249f36c1a403, user_id=b872ce80cf3147ea8f31d63a137b3250, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:55.471298 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f0d52-ca50-42b9-af64-738add614fbd admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:55.512053 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 927/1835] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:12:55 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:55.515866 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b55d6b-36b1-4694-a3b9-9ab9742909c0 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:55.532331 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 909/1836] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:55 2022] PUT /identity/v3/projects/7fa5862a888d42fc8f85249f36c1a403/users/b872ce80cf3147ea8f31d63a137b3250/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:55.545092 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f0d52-ca50-42b9-af64-738add614fbd admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:55.549626 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:55.691383 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b55d6b-36b1-4694-a3b9-9ab9742909c0 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:55.775517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f0d52-ca50-42b9-af64-738add614fbd admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:55.805956 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185a2ae-e7c1-4c4c-938f-b54a412678c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:55.806850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185a2ae-e7c1-4c4c-938f-b54a412678c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:55.808733 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185a2ae-e7c1-4c4c-938f-b54a412678c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:55.809593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185a2ae-e7c1-4c4c-938f-b54a412678c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:55.812987 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a185a2ae-e7c1-4c4c-938f-b54a412678c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:55.819591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a185a2ae-e7c1-4c4c-938f-b54a412678c2 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:55.826812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ac8db-eabf-43a5-9672-2304d76ded67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:55.827489 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ac8db-eabf-43a5-9672-2304d76ded67 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:55.827765 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ac8db-eabf-43a5-9672-2304d76ded67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:55.828033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ac8db-eabf-43a5-9672-2304d76ded67 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:55.828679 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ac8db-eabf-43a5-9672-2304d76ded67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:55.833161 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ac8db-eabf-43a5-9672-2304d76ded67 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:55.847906 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 910/1837] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:55.860470 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185a2ae-e7c1-4c4c-938f-b54a412678c2 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:55.880054 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 928/1838] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:12:55 2022] POST /identity/v3/users => generated 293 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:55.884913 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ac8db-eabf-43a5-9672-2304d76ded67 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:55.893662 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185a2ae-e7c1-4c4c-938f-b54a412678c2 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:55.905216 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ac8db-eabf-43a5-9672-2304d76ded67 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:55.906238 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e437c46b-ec66-427e-836f-57c0b43c089f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:55.907205 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437c46b-ec66-427e-836f-57c0b43c089f admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:55.907597 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437c46b-ec66-427e-836f-57c0b43c089f admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:55.908007 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437c46b-ec66-427e-836f-57c0b43c089f admin admin] PATH_INFO: `/v3/projects/7fa5862a888d42fc8f85249f36c1a403/users/b872ce80cf3147ea8f31d63a137b3250/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:55.910799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e437c46b-ec66-427e-836f-57c0b43c089f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fa5862a888d42fc8f85249f36c1a403, user_id=b872ce80cf3147ea8f31d63a137b3250, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:55.916815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10807b3a-c56d-49d4-8b59-5bef9639a354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:55.917510 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10807b3a-c56d-49d4-8b59-5bef9639a354 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:55.917790 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10807b3a-c56d-49d4-8b59-5bef9639a354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:55.918060 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10807b3a-c56d-49d4-8b59-5bef9639a354 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:55.918638 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e437c46b-ec66-427e-836f-57c0b43c089f admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:55.918768 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10807b3a-c56d-49d4-8b59-5bef9639a354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:55.922608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10807b3a-c56d-49d4-8b59-5bef9639a354 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:55.944916 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 929/1840] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:12:55.949445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10807b3a-c56d-49d4-8b59-5bef9639a354 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:55.957033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 911/1840] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:55 2022] PUT /identity/v3/projects/7fa5862a888d42fc8f85249f36c1a403/users/b872ce80cf3147ea8f31d63a137b3250/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:55.958661 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1982598-37cb-4bfe-9793-09d1e1b84520 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:55.959309 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1982598-37cb-4bfe-9793-09d1e1b84520 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:55.959600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1982598-37cb-4bfe-9793-09d1e1b84520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:55.974531 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10807b3a-c56d-49d4-8b59-5bef9639a354 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:56.014889 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-a1982598-37cb-4bfe-9793-09d1e1b84520 None None] MFA Rules not processed for user `b872ce80cf3147ea8f31d63a137b3250`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:56.017484 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-a1982598-37cb-4bfe-9793-09d1e1b84520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:56.091266 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:56.112076 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347fe27-64d0-4275-a59d-1b7be13d1d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:56.113752 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347fe27-64d0-4275-a59d-1b7be13d1d02 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:56.114596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347fe27-64d0-4275-a59d-1b7be13d1d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:56.115479 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347fe27-64d0-4275-a59d-1b7be13d1d02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:56.116934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347fe27-64d0-4275-a59d-1b7be13d1d02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:56.118916 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347fe27-64d0-4275-a59d-1b7be13d1d02 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:56.161041 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 930/1841] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:56.164787 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347fe27-64d0-4275-a59d-1b7be13d1d02 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:56.197903 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347fe27-64d0-4275-a59d-1b7be13d1d02 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:56.217104 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3cf0b2-7d18-43e1-aa21-115f8e538093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:56.218587 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3cf0b2-7d18-43e1-aa21-115f8e538093 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:56.220190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3cf0b2-7d18-43e1-aa21-115f8e538093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:56.222598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3cf0b2-7d18-43e1-aa21-115f8e538093 admin admin] PATH_INFO: `/v3/projects/564ab122f1c0461987ab59f6d259b769/users/8161565be48d4c32b0efc3d287b286a0/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:56.246301 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 912/1842] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:55 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:56.265000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1982598-37cb-4bfe-9793-09d1e1b84520 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:56.267750 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-a1982598-37cb-4bfe-9793-09d1e1b84520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:56.295095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3cf0b2-7d18-43e1-aa21-115f8e538093 admin admin] RBAC: Authorizing `identity:create_grant(project_id=564ab122f1c0461987ab59f6d259b769, user_id=8161565be48d4c32b0efc3d287b286a0, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:56.300104 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3cf0b2-7d18-43e1-aa21-115f8e538093 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:56.328198 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 931/1843] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:56 2022] PUT /identity/v3/projects/564ab122f1c0461987ab59f6d259b769/users/8161565be48d4c32b0efc3d287b286a0/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:56.343656 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db7ef5-7054-429f-918e-854a0539c16c None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:56.344572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db7ef5-7054-429f-918e-854a0539c16c None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:56.345059 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db7ef5-7054-429f-918e-854a0539c16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:56.410510 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:56.451121 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-b3db7ef5-7054-429f-918e-854a0539c16c None None] MFA Rules not processed for user `8161565be48d4c32b0efc3d287b286a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:56.459263 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-b3db7ef5-7054-429f-918e-854a0539c16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:56.547279 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1982598-37cb-4bfe-9793-09d1e1b84520 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:56.730362 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 932/1844] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:56.733341 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db7ef5-7054-429f-918e-854a0539c16c None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:56.740984 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5c55d-e5d5-4392-bb55-c65fdb2e1bb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:56.742822 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5c55d-e5d5-4392-bb55-c65fdb2e1bb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:56.743417 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5c55d-e5d5-4392-bb55-c65fdb2e1bb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:56.743922 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5c55d-e5d5-4392-bb55-c65fdb2e1bb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:56.907158 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db7ef5-7054-429f-918e-854a0539c16c None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:56.924727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f763fa9-e52d-4bb3-bf74-f2bb914b9bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:56.925537 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f763fa9-e52d-4bb3-bf74-f2bb914b9bd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:56.925908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f763fa9-e52d-4bb3-bf74-f2bb914b9bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:56.926279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f763fa9-e52d-4bb3-bf74-f2bb914b9bd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:56.927102 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f763fa9-e52d-4bb3-bf74-f2bb914b9bd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:56.929540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f763fa9-e52d-4bb3-bf74-f2bb914b9bd6 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:57.006555 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 933/1845] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:12:56 2022] POST /identity/v3/projects => generated 383 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:57.011630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f763fa9-e52d-4bb3-bf74-f2bb914b9bd6 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:57.041049 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d5c55d-e5d5-4392-bb55-c65fdb2e1bb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:57.042723 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d5c55d-e5d5-4392-bb55-c65fdb2e1bb7 service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:57.067693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 913/1846] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:56 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 803 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:57.324499 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5c55d-e5d5-4392-bb55-c65fdb2e1bb7 service cinder] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:57.328466 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f763fa9-e52d-4bb3-bf74-f2bb914b9bd6 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:57.335420 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5c55d-e5d5-4392-bb55-c65fdb2e1bb7 service cinder] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:57.337827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa7849-ef17-4ce0-9670-94465c33dfa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:57.339761 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa7849-ef17-4ce0-9670-94465c33dfa4 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:57.340249 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa7849-ef17-4ce0-9670-94465c33dfa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:57.340743 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa7849-ef17-4ce0-9670-94465c33dfa4 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:57.343783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa7849-ef17-4ce0-9670-94465c33dfa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:57.349903 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa7849-ef17-4ce0-9670-94465c33dfa4 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:57.427547 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 934/1847] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:12:57 2022] POST /identity/v3/users => generated 292 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:57.428696 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa7849-ef17-4ce0-9670-94465c33dfa4 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:57.437318 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-65fa7849-ef17-4ce0-9670-94465c33dfa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:57.484828 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:57.502501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:57.588203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1b86f-438d-4e08-87ac-de1b1b602464 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:57.588978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1b86f-438d-4e08-87ac-de1b1b602464 service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:57.589080 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1b86f-438d-4e08-87ac-de1b1b602464 service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:57.589399 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1b86f-438d-4e08-87ac-de1b1b602464 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:57.592375 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f9e1b86f-438d-4e08-87ac-de1b1b602464 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:57.655438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa7849-ef17-4ce0-9670-94465c33dfa4 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:57.670046 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9b46a-956f-4cf4-a96b-e65ddaab6ccb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:57.673105 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9b46a-956f-4cf4-a96b-e65ddaab6ccb service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:57.673585 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9b46a-956f-4cf4-a96b-e65ddaab6ccb service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:57.674141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9b46a-956f-4cf4-a96b-e65ddaab6ccb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:57.683177 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-e3d9b46a-956f-4cf4-a96b-e65ddaab6ccb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:57.843604 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e1b86f-438d-4e08-87ac-de1b1b602464 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:57.845044 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d9b46a-956f-4cf4-a96b-e65ddaab6ccb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:57.846674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d9b46a-956f-4cf4-a96b-e65ddaab6ccb service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:57.849149 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e1b86f-438d-4e08-87ac-de1b1b602464 service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:57.871066 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 935/1848] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:12:57.885004 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9b46a-956f-4cf4-a96b-e65ddaab6ccb service cinder] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:57.905412 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 914/1849] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:12:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:12:57.911053 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9b46a-956f-4cf4-a96b-e65ddaab6ccb service cinder] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:57.918028 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744138d5-cf58-4ddc-8555-162da0db4686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:57.918765 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744138d5-cf58-4ddc-8555-162da0db4686 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:57.919727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744138d5-cf58-4ddc-8555-162da0db4686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:57.920027 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744138d5-cf58-4ddc-8555-162da0db4686 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:57.920806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744138d5-cf58-4ddc-8555-162da0db4686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:57.922939 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744138d5-cf58-4ddc-8555-162da0db4686 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:57.973803 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 936/1850] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:57.975001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1b86f-438d-4e08-87ac-de1b1b602464 service cinder] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:58.004149 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1b86f-438d-4e08-87ac-de1b1b602464 service cinder] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:58.035462 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29cd8d4-517d-4257-8b36-4fcf171a1260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:58.036116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cd8d4-517d-4257-8b36-4fcf171a1260 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:58.037697 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cd8d4-517d-4257-8b36-4fcf171a1260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:58.037993 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cd8d4-517d-4257-8b36-4fcf171a1260 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:58.040844 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29cd8d4-517d-4257-8b36-4fcf171a1260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:58.047827 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29cd8d4-517d-4257-8b36-4fcf171a1260 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:58.096689 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 915/1851] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:58.102474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744138d5-cf58-4ddc-8555-162da0db4686 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:58.130604 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744138d5-cf58-4ddc-8555-162da0db4686 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:58.139295 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c8b51-ff3c-487e-badc-f41f564e6368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:58.141559 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c8b51-ff3c-487e-badc-f41f564e6368 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:58.141846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c8b51-ff3c-487e-badc-f41f564e6368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:58.142998 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c8b51-ff3c-487e-badc-f41f564e6368 admin admin] PATH_INFO: `/v3/projects/f361435efba64cf1a832bd250dcc59ef/users/83e6a62d72d54fbdac8815d1cb94d9a0/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:58.237834 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c8b51-ff3c-487e-badc-f41f564e6368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f361435efba64cf1a832bd250dcc59ef, user_id=83e6a62d72d54fbdac8815d1cb94d9a0, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:58.257819 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c8b51-ff3c-487e-badc-f41f564e6368 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:58.309587 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 937/1852] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:58 2022] PUT /identity/v3/projects/f361435efba64cf1a832bd250dcc59ef/users/83e6a62d72d54fbdac8815d1cb94d9a0/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:58.325803 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f7336-8092-4798-9016-4d2071063392 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:58.326647 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f7336-8092-4798-9016-4d2071063392 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:58.327531 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f7336-8092-4798-9016-4d2071063392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:58.431732 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-9f0f7336-8092-4798-9016-4d2071063392 None None] MFA Rules not processed for user `83e6a62d72d54fbdac8815d1cb94d9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:12:58.436399 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-9f0f7336-8092-4798-9016-4d2071063392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:58.631579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 916/1853] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:12:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:12:58.636977 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c8b51-ff3c-487e-badc-f41f564e6368 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:58.868625 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c8b51-ff3c-487e-badc-f41f564e6368 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:58.883785 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c1c86-098f-44e3-957e-84ac10ba793e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:58.885330 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c1c86-098f-44e3-957e-84ac10ba793e admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:58.885616 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c1c86-098f-44e3-957e-84ac10ba793e admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:58.885888 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c1c86-098f-44e3-957e-84ac10ba793e admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:58.886556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c1c86-098f-44e3-957e-84ac10ba793e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:58.890018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c1c86-098f-44e3-957e-84ac10ba793e admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:58.981363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 938/1854] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:12:58 2022] POST /identity/v3/projects => generated 385 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:58.984703 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f7336-8092-4798-9016-4d2071063392 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:59.008631 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f7336-8092-4798-9016-4d2071063392 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:59.345638 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7636a8-9d1c-4641-a0f1-57471adabd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:59.352391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7636a8-9d1c-4641-a0f1-57471adabd37 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:59.355312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7636a8-9d1c-4641-a0f1-57471adabd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:59.355794 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7636a8-9d1c-4641-a0f1-57471adabd37 admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:59.360720 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7636a8-9d1c-4641-a0f1-57471adabd37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:59.377472 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7636a8-9d1c-4641-a0f1-57471adabd37 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:59.434596 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:59.446625 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c1c86-098f-44e3-957e-84ac10ba793e admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:59.453070 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 917/1855] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:12:58 2022] POST /identity/v3/users => generated 299 bytes in 471 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:12:59.463890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c1c86-098f-44e3-957e-84ac10ba793e admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:59.475105 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7636a8-9d1c-4641-a0f1-57471adabd37 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:59.477446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-ef7636a8-9d1c-4641-a0f1-57471adabd37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:59.486067 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45a1ca-9d75-4dbe-8e1c-2ebcbd1989f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:59.487080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45a1ca-9d75-4dbe-8e1c-2ebcbd1989f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:59.487551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45a1ca-9d75-4dbe-8e1c-2ebcbd1989f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:59.488013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45a1ca-9d75-4dbe-8e1c-2ebcbd1989f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:59.488897 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba45a1ca-9d75-4dbe-8e1c-2ebcbd1989f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:59.491956 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba45a1ca-9d75-4dbe-8e1c-2ebcbd1989f5 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:59.510743 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7636a8-9d1c-4641-a0f1-57471adabd37 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:59.530962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acc0d3-7e8e-47cc-8644-482a5961dea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:59.531992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acc0d3-7e8e-47cc-8644-482a5961dea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:59.534104 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acc0d3-7e8e-47cc-8644-482a5961dea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:59.537740 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acc0d3-7e8e-47cc-8644-482a5961dea2 admin admin] PATH_INFO: `/v3/projects/97eead5cc7a4450b8a3477e917d434ea` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:59.539823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4acc0d3-7e8e-47cc-8644-482a5961dea2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97eead5cc7a4450b8a3477e917d434ea)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:59.540050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 939/1856] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:12:59.541235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45a1ca-9d75-4dbe-8e1c-2ebcbd1989f5 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:59.548465 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4acc0d3-7e8e-47cc-8644-482a5961dea2 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:59.579343 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45a1ca-9d75-4dbe-8e1c-2ebcbd1989f5 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:59.593688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee61949d-482a-4ab0-b67b-34fe5c72d3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:59.597667 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61949d-482a-4ab0-b67b-34fe5c72d3a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:59.598140 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61949d-482a-4ab0-b67b-34fe5c72d3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:59.599680 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61949d-482a-4ab0-b67b-34fe5c72d3a8 admin admin] PATH_INFO: `/v3/projects/ef579a625f6a46b39b8c21fabed3eb7c/users/00fa577ef860438e8795b9ab3424205b/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:59.610782 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:12:59.656133 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee61949d-482a-4ab0-b67b-34fe5c72d3a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef579a625f6a46b39b8c21fabed3eb7c, user_id=00fa577ef860438e8795b9ab3424205b, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:59.663725 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee61949d-482a-4ab0-b67b-34fe5c72d3a8 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:12:59.716855 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 940/1857] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:12:59 2022] PUT /identity/v3/projects/ef579a625f6a46b39b8c21fabed3eb7c/users/00fa577ef860438e8795b9ab3424205b/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:12:59.721900 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee61949d-482a-4ab0-b67b-34fe5c72d3a8 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:59.726818 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c4acc0d3-7e8e-47cc-8644-482a5961dea2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97eead5cc7a4450b8a3477e917d434ea', 'request_id': 'req-c4acc0d3-7e8e-47cc-8644-482a5961dea2'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:59.752480 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c4acc0d3-7e8e-47cc-8644-482a5961dea2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97eead5cc7a4450b8a3477e917d434ea', 'request_id': 'req-c4acc0d3-7e8e-47cc-8644-482a5961dea2'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:59.771862 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c4acc0d3-7e8e-47cc-8644-482a5961dea2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97eead5cc7a4450b8a3477e917d434ea', 'request_id': 'req-c4acc0d3-7e8e-47cc-8644-482a5961dea2'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:59.807738 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c4acc0d3-7e8e-47cc-8644-482a5961dea2 admin admin] The token cache is being invalidate because project 97eead5cc7a4450b8a3477e917d434ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:12:59.808035 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c4acc0d3-7e8e-47cc-8644-482a5961dea2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:59.813535 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c4acc0d3-7e8e-47cc-8644-482a5961dea2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:12:59.815494 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 918/1858] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:12:59 2022] DELETE /identity/v3/projects/97eead5cc7a4450b8a3477e917d434ea => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:12:59.819654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acc0d3-7e8e-47cc-8644-482a5961dea2 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:12:59.821095 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c4acc0d3-7e8e-47cc-8644-482a5961dea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:12:59.962474 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acc0d3-7e8e-47cc-8644-482a5961dea2 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:12:59.976489 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3970e-a790-4204-8866-d233099f81eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:12:59.977483 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3970e-a790-4204-8866-d233099f81eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:12:59.977898 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3970e-a790-4204-8866-d233099f81eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:12:59.978443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3970e-a790-4204-8866-d233099f81eb admin admin] PATH_INFO: `/v3/users/78d03ccac11a452b8f3fdb455c659751` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:12:59.979838 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f3970e-a790-4204-8866-d233099f81eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=78d03ccac11a452b8f3fdb455c659751)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:12:59.982042 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f3970e-a790-4204-8866-d233099f81eb admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:00.006592 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee61949d-482a-4ab0-b67b-34fe5c72d3a8 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:00.027375 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725a725-9e41-4f34-96c9-2028dc728058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:00.029377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725a725-9e41-4f34-96c9-2028dc728058 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:00.029377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725a725-9e41-4f34-96c9-2028dc728058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:00.029377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725a725-9e41-4f34-96c9-2028dc728058 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:00.039774 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725a725-9e41-4f34-96c9-2028dc728058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:00.043958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725a725-9e41-4f34-96c9-2028dc728058 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:00.098006 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 941/1859] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:00.101307 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725a725-9e41-4f34-96c9-2028dc728058 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:00.108681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-7725a725-9e41-4f34-96c9-2028dc728058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:00.120405 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-26f3970e-a790-4204-8866-d233099f81eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78d03ccac11a452b8f3fdb455c659751'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:00.136077 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-26f3970e-a790-4204-8866-d233099f81eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78d03ccac11a452b8f3fdb455c659751'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:00.168834 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725a725-9e41-4f34-96c9-2028dc728058 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:00.181352 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3e0a8-ee2f-4e90-8aab-46200ef4c9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:00.182376 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3e0a8-ee2f-4e90-8aab-46200ef4c9fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:00.182662 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3e0a8-ee2f-4e90-8aab-46200ef4c9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:00.182959 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3e0a8-ee2f-4e90-8aab-46200ef4c9fc admin admin] PATH_INFO: `/v3/projects/ef579a625f6a46b39b8c21fabed3eb7c/users/00fa577ef860438e8795b9ab3424205b/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:00.201365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3e0a8-ee2f-4e90-8aab-46200ef4c9fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef579a625f6a46b39b8c21fabed3eb7c, user_id=00fa577ef860438e8795b9ab3424205b, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:00.203951 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-26f3970e-a790-4204-8866-d233099f81eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78d03ccac11a452b8f3fdb455c659751'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:00.205826 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-26f3970e-a790-4204-8866-d233099f81eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78d03ccac11a452b8f3fdb455c659751'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:00.215753 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3e0a8-ee2f-4e90-8aab-46200ef4c9fc admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:00.230085 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-26f3970e-a790-4204-8866-d233099f81eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78d03ccac11a452b8f3fdb455c659751'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:00.248567 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 919/1860] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:12:59 2022] DELETE /identity/v3/users/78d03ccac11a452b8f3fdb455c659751 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:13:00.263096 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 942/1861] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:00 2022] PUT /identity/v3/projects/ef579a625f6a46b39b8c21fabed3eb7c/users/00fa577ef860438e8795b9ab3424205b/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:00.277762 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6dfc6-25f4-412d-bd6d-cb31b7b0174a None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:00.278758 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6dfc6-25f4-412d-bd6d-cb31b7b0174a None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:00.279176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6dfc6-25f4-412d-bd6d-cb31b7b0174a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:00.290819 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:00.391962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-98d6dfc6-25f4-412d-bd6d-cb31b7b0174a None None] MFA Rules not processed for user `00fa577ef860438e8795b9ab3424205b`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:00.395035 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-98d6dfc6-25f4-412d-bd6d-cb31b7b0174a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:00.659466 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 920/1862] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:00.667228 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3e0a8-ee2f-4e90-8aab-46200ef4c9fc admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:00.669218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-b4f3e0a8-ee2f-4e90-8aab-46200ef4c9fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:00.837528 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3e0a8-ee2f-4e90-8aab-46200ef4c9fc admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:00.995332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea202584-3833-4e16-aefe-47b6b1e1829b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:00.996409 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea202584-3833-4e16-aefe-47b6b1e1829b service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:00.996820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea202584-3833-4e16-aefe-47b6b1e1829b service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:00.997238 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea202584-3833-4e16-aefe-47b6b1e1829b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:01.000763 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-ea202584-3833-4e16-aefe-47b6b1e1829b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:01.100300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:01.151764 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea202584-3833-4e16-aefe-47b6b1e1829b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:01.153758 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea202584-3833-4e16-aefe-47b6b1e1829b service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:01.166621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 943/1863] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:01.248428 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b13bf-93d6-4fbe-8a0b-9c92350e6ce8 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:01.249012 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b13bf-93d6-4fbe-8a0b-9c92350e6ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:01.249472 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b13bf-93d6-4fbe-8a0b-9c92350e6ce8 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:01.251411 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 921/1864] 10.222.0.44 () {54 vars in 921 bytes} [Sun Sep 11 15:13:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:13:01.261794 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67f9c6-8af8-4452-9591-d133e8e96779 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:01.264054 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67f9c6-8af8-4452-9591-d133e8e96779 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:01.264054 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67f9c6-8af8-4452-9591-d133e8e96779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:01.320930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-0a67f9c6-8af8-4452-9591-d133e8e96779 None None] MFA Rules not processed for user `453297dac64046d19fc1c331d2d85abc`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:01.322283 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-0a67f9c6-8af8-4452-9591-d133e8e96779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:01.410513 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 944/1865] 10.222.0.44 () {58 vars in 1018 bytes} [Sun Sep 11 15:13:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:01.785586 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b13bf-93d6-4fbe-8a0b-9c92350e6ce8 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:01.792017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-9d2b13bf-93d6-4fbe-8a0b-9c92350e6ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:01.815078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b13bf-93d6-4fbe-8a0b-9c92350e6ce8 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:01.822779 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead61f5e-f282-4f85-8c01-8c919a911597 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:01.823718 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead61f5e-f282-4f85-8c01-8c919a911597 service nova] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:01.824166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead61f5e-f282-4f85-8c01-8c919a911597 service nova] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:01.825077 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead61f5e-f282-4f85-8c01-8c919a911597 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:01.837398 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead61f5e-f282-4f85-8c01-8c919a911597 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:01.839849 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead61f5e-f282-4f85-8c01-8c919a911597 service nova] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:01.877291 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 922/1866] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:01.995099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67f9c6-8af8-4452-9591-d133e8e96779 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:01.996576 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-0a67f9c6-8af8-4452-9591-d133e8e96779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:02.090087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead61f5e-f282-4f85-8c01-8c919a911597 service nova] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:02.092756 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-ead61f5e-f282-4f85-8c01-8c919a911597 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:02.161634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67f9c6-8af8-4452-9591-d133e8e96779 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:02.171540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97190eb8-9bfc-499f-988c-d5feac4b529d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:02.172473 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97190eb8-9bfc-499f-988c-d5feac4b529d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:02.173476 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97190eb8-9bfc-499f-988c-d5feac4b529d admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:02.173890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97190eb8-9bfc-499f-988c-d5feac4b529d admin admin] PATH_INFO: `/v3/users/794f74d3cc14458aaa28621128f699b6` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:02.178025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97190eb8-9bfc-499f-988c-d5feac4b529d admin admin] RBAC: Authorizing `identity:delete_user(user_id=794f74d3cc14458aaa28621128f699b6)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:02.180411 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97190eb8-9bfc-499f-988c-d5feac4b529d admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:02.285952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead61f5e-f282-4f85-8c01-8c919a911597 service nova] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:02.288476 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-97190eb8-9bfc-499f-988c-d5feac4b529d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '794f74d3cc14458aaa28621128f699b6'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:02.298114 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419227f3-22d3-47ee-b3c9-f753a4bfa5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:02.299227 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419227f3-22d3-47ee-b3c9-f753a4bfa5f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:02.299658 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419227f3-22d3-47ee-b3c9-f753a4bfa5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:02.300676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419227f3-22d3-47ee-b3c9-f753a4bfa5f7 admin admin] PATH_INFO: `/v3/projects/a840430652fd435bbe83189ad842b751` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:02.302274 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419227f3-22d3-47ee-b3c9-f753a4bfa5f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a840430652fd435bbe83189ad842b751)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:02.304845 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-97190eb8-9bfc-499f-988c-d5feac4b529d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '794f74d3cc14458aaa28621128f699b6'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:02.307230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419227f3-22d3-47ee-b3c9-f753a4bfa5f7 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:02.327067 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-97190eb8-9bfc-499f-988c-d5feac4b529d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '794f74d3cc14458aaa28621128f699b6'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:02.328726 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-97190eb8-9bfc-499f-988c-d5feac4b529d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '794f74d3cc14458aaa28621128f699b6'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:02.331998 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:02.346397 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-97190eb8-9bfc-499f-988c-d5feac4b529d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '794f74d3cc14458aaa28621128f699b6'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:02.362909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 945/1867] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:01 2022] DELETE /identity/v3/users/794f74d3cc14458aaa28621128f699b6 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:02.368788 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48485d39-81a7-4f67-9dd3-53299aaabbe3 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:02.371060 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48485d39-81a7-4f67-9dd3-53299aaabbe3 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:02.371406 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48485d39-81a7-4f67-9dd3-53299aaabbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:02.404919 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-48485d39-81a7-4f67-9dd3-53299aaabbe3 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:02.406718 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-48485d39-81a7-4f67-9dd3-53299aaabbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:02.409295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-419227f3-22d3-47ee-b3c9-f753a4bfa5f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a840430652fd435bbe83189ad842b751', 'request_id': 'req-419227f3-22d3-47ee-b3c9-f753a4bfa5f7'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:02.431688 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-419227f3-22d3-47ee-b3c9-f753a4bfa5f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a840430652fd435bbe83189ad842b751', 'request_id': 'req-419227f3-22d3-47ee-b3c9-f753a4bfa5f7'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:02.444839 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-419227f3-22d3-47ee-b3c9-f753a4bfa5f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a840430652fd435bbe83189ad842b751', 'request_id': 'req-419227f3-22d3-47ee-b3c9-f753a4bfa5f7'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:02.480452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-419227f3-22d3-47ee-b3c9-f753a4bfa5f7 admin admin] The token cache is being invalidate because project a840430652fd435bbe83189ad842b751 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:02.480452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-419227f3-22d3-47ee-b3c9-f753a4bfa5f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:02.482254 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-419227f3-22d3-47ee-b3c9-f753a4bfa5f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:02.493543 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 923/1868] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:02 2022] DELETE /identity/v3/projects/a840430652fd435bbe83189ad842b751 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:02.498400 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54bb40-1030-4344-9d4c-c04f31c00b0c None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:02.498731 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54bb40-1030-4344-9d4c-c04f31c00b0c None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:02.498910 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54bb40-1030-4344-9d4c-c04f31c00b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:02.508581 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:02.524769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-aa54bb40-1030-4344-9d4c-c04f31c00b0c None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:02.527433 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-aa54bb40-1030-4344-9d4c-c04f31c00b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:02.596218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 946/1869] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:02.618163 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48485d39-81a7-4f67-9dd3-53299aaabbe3 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:02.620149 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-48485d39-81a7-4f67-9dd3-53299aaabbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:02.626594 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 924/1870] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:02.634745 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a3c14-0d5a-4952-9c7c-9850333457ef None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:02.635032 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a3c14-0d5a-4952-9c7c-9850333457ef None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:02.635304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a3c14-0d5a-4952-9c7c-9850333457ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:02.700429 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-ca6a3c14-0d5a-4952-9c7c-9850333457ef None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:02.701658 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-ca6a3c14-0d5a-4952-9c7c-9850333457ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:02.724514 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 925/1871] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:02.730564 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a3c14-0d5a-4952-9c7c-9850333457ef None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:02.745097 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a3c14-0d5a-4952-9c7c-9850333457ef None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:02.751805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00766f60-f384-4a5b-a96e-cb90caa86b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:02.752534 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00766f60-f384-4a5b-a96e-cb90caa86b03 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:02.752904 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00766f60-f384-4a5b-a96e-cb90caa86b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:02.753224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00766f60-f384-4a5b-a96e-cb90caa86b03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:02.754175 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00766f60-f384-4a5b-a96e-cb90caa86b03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:13:02.754780 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00766f60-f384-4a5b-a96e-cb90caa86b03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:02.756952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00766f60-f384-4a5b-a96e-cb90caa86b03 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:02.789399 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48485d39-81a7-4f67-9dd3-53299aaabbe3 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:02.794845 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634917b7-0848-45d5-92e8-8ffb7cd50d01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:02.798784 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634917b7-0848-45d5-92e8-8ffb7cd50d01 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:02.799419 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634917b7-0848-45d5-92e8-8ffb7cd50d01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:02.799640 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 926/1872] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:13:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:13:02.800009 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634917b7-0848-45d5-92e8-8ffb7cd50d01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:02.801020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00766f60-f384-4a5b-a96e-cb90caa86b03 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:02.814475 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634917b7-0848-45d5-92e8-8ffb7cd50d01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:02.816153 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00766f60-f384-4a5b-a96e-cb90caa86b03 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:02.817124 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634917b7-0848-45d5-92e8-8ffb7cd50d01 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:02.823725 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea0653-9ab2-48e9-ab0e-9f1574768a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:02.825842 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea0653-9ab2-48e9-ab0e-9f1574768a81 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:02.827305 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea0653-9ab2-48e9-ab0e-9f1574768a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:02.828497 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea0653-9ab2-48e9-ab0e-9f1574768a81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:02.829508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea0653-9ab2-48e9-ab0e-9f1574768a81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:02.832443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea0653-9ab2-48e9-ab0e-9f1574768a81 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:02.834550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 947/1873] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:02.879140 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 927/1874] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:13:02 2022] POST /identity/v3/projects => generated 381 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:02.884960 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634917b7-0848-45d5-92e8-8ffb7cd50d01 service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:02.911582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634917b7-0848-45d5-92e8-8ffb7cd50d01 service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:03.024667 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b7e1d-c5fa-4f8a-8659-1efcfd5d3bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:03.025658 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b7e1d-c5fa-4f8a-8659-1efcfd5d3bb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:03.026356 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b7e1d-c5fa-4f8a-8659-1efcfd5d3bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:03.026900 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b7e1d-c5fa-4f8a-8659-1efcfd5d3bb2 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:03.027958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b7e1d-c5fa-4f8a-8659-1efcfd5d3bb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:03.030564 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b7e1d-c5fa-4f8a-8659-1efcfd5d3bb2 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:03.087933 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 948/1875] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:13:02 2022] POST /identity/v3/users => generated 291 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:03.092788 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea0653-9ab2-48e9-ab0e-9f1574768a81 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:03.108606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea0653-9ab2-48e9-ab0e-9f1574768a81 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:03.114608 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28598456-fbc7-451d-a9f3-bf1522ae5536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:03.115453 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28598456-fbc7-451d-a9f3-bf1522ae5536 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:03.115828 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28598456-fbc7-451d-a9f3-bf1522ae5536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:03.117346 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28598456-fbc7-451d-a9f3-bf1522ae5536 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:03.118139 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28598456-fbc7-451d-a9f3-bf1522ae5536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:03.120123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28598456-fbc7-451d-a9f3-bf1522ae5536 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:03.134879 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 928/1876] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:03.138856 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b7e1d-c5fa-4f8a-8659-1efcfd5d3bb2 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:03.149535 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b7e1d-c5fa-4f8a-8659-1efcfd5d3bb2 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:03.150979 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28598456-fbc7-451d-a9f3-bf1522ae5536 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:03.153224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-28598456-fbc7-451d-a9f3-bf1522ae5536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:03.172223 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4244c4b-0e51-4463-8888-5b4167eae9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:03.172879 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4244c4b-0e51-4463-8888-5b4167eae9ab admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:03.173125 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4244c4b-0e51-4463-8888-5b4167eae9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:03.173522 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4244c4b-0e51-4463-8888-5b4167eae9ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:03.174188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4244c4b-0e51-4463-8888-5b4167eae9ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:03.175829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4244c4b-0e51-4463-8888-5b4167eae9ab admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:03.191447 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 949/1877] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:03.192378 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4244c4b-0e51-4463-8888-5b4167eae9ab admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:03.208123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4244c4b-0e51-4463-8888-5b4167eae9ab admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:03.220559 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2871540-97cc-4614-9680-83c6deae5a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:03.221690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2871540-97cc-4614-9680-83c6deae5a1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:03.221690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2871540-97cc-4614-9680-83c6deae5a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:03.221870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2871540-97cc-4614-9680-83c6deae5a1a admin admin] PATH_INFO: `/v3/projects/99c1fddc22af4a2c912110853edd6316/users/7ef1b695e6fd4f98a98464512cc042d5/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:03.226250 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28598456-fbc7-451d-a9f3-bf1522ae5536 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:03.230182 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d05ea-849e-4b74-bcfe-1a9787f71555 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:03.230977 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d05ea-849e-4b74-bcfe-1a9787f71555 service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:03.231321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d05ea-849e-4b74-bcfe-1a9787f71555 service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:03.231658 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d05ea-849e-4b74-bcfe-1a9787f71555 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:03.233975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c05d05ea-849e-4b74-bcfe-1a9787f71555 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:03.241599 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:03.254788 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2871540-97cc-4614-9680-83c6deae5a1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=99c1fddc22af4a2c912110853edd6316, user_id=7ef1b695e6fd4f98a98464512cc042d5, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:03.258651 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2871540-97cc-4614-9680-83c6deae5a1a admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:03.292636 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 950/1878] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:03 2022] PUT /identity/v3/projects/99c1fddc22af4a2c912110853edd6316/users/7ef1b695e6fd4f98a98464512cc042d5/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:13:03.298779 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f859f74-4676-49b1-97e4-d182326161e4 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:03.299948 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f859f74-4676-49b1-97e4-d182326161e4 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:03.300449 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f859f74-4676-49b1-97e4-d182326161e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:03.359410 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-4f859f74-4676-49b1-97e4-d182326161e4 None None] MFA Rules not processed for user `7ef1b695e6fd4f98a98464512cc042d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:03.361061 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-4f859f74-4676-49b1-97e4-d182326161e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:03.373427 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d05ea-849e-4b74-bcfe-1a9787f71555 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:03.375052 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d05ea-849e-4b74-bcfe-1a9787f71555 service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:03.392549 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 929/1879] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:03.488101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 951/1880] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:03.491905 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d05ea-849e-4b74-bcfe-1a9787f71555 service cinder] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:03.504810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d05ea-849e-4b74-bcfe-1a9787f71555 service cinder] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:03.510279 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:03.667562 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad102db-5837-435a-b484-e49332034312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:03.668466 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad102db-5837-435a-b484-e49332034312 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:03.668875 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad102db-5837-435a-b484-e49332034312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:03.669313 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad102db-5837-435a-b484-e49332034312 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:03.672406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad102db-5837-435a-b484-e49332034312 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:03.674973 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad102db-5837-435a-b484-e49332034312 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:03.676309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f859f74-4676-49b1-97e4-d182326161e4 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:03.678517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-4f859f74-4676-49b1-97e4-d182326161e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:03.703706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f859f74-4676-49b1-97e4-d182326161e4 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:03.719294 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b7137-286a-4e1a-a37e-670d7f9e2059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:03.720065 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b7137-286a-4e1a-a37e-670d7f9e2059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:03.720413 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b7137-286a-4e1a-a37e-670d7f9e2059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:03.720825 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b7137-286a-4e1a-a37e-670d7f9e2059 admin admin] PATH_INFO: `/v3/projects/881ee6aa169c4dc7bb8fad51170e7e30` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:03.722632 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b7137-286a-4e1a-a37e-670d7f9e2059 admin admin] RBAC: Authorizing `identity:delete_project(project_id=881ee6aa169c4dc7bb8fad51170e7e30)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:03.725023 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b7137-286a-4e1a-a37e-670d7f9e2059 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:03.733498 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 930/1881] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:13:03 2022] POST /identity/v3/projects => generated 379 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:03.737221 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad102db-5837-435a-b484-e49332034312 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:03.809064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-a27b7137-286a-4e1a-a37e-670d7f9e2059 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '881ee6aa169c4dc7bb8fad51170e7e30', 'request_id': 'req-a27b7137-286a-4e1a-a37e-670d7f9e2059'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:03.815634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-a27b7137-286a-4e1a-a37e-670d7f9e2059 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '881ee6aa169c4dc7bb8fad51170e7e30', 'request_id': 'req-a27b7137-286a-4e1a-a37e-670d7f9e2059'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:03.822174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-a27b7137-286a-4e1a-a37e-670d7f9e2059 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '881ee6aa169c4dc7bb8fad51170e7e30', 'request_id': 'req-a27b7137-286a-4e1a-a37e-670d7f9e2059'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:03.828262 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad102db-5837-435a-b484-e49332034312 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:03.839135 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-a27b7137-286a-4e1a-a37e-670d7f9e2059 admin admin] The token cache is being invalidate because project 881ee6aa169c4dc7bb8fad51170e7e30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:03.839430 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-a27b7137-286a-4e1a-a37e-670d7f9e2059 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:03.840349 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-a27b7137-286a-4e1a-a37e-670d7f9e2059 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:03.842283 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 952/1882] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:03 2022] DELETE /identity/v3/projects/881ee6aa169c4dc7bb8fad51170e7e30 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:03.846585 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b7137-286a-4e1a-a37e-670d7f9e2059 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:03.848361 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-a27b7137-286a-4e1a-a37e-670d7f9e2059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:03.891226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81ca22-ce8a-42bb-84ef-bfced48db7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:03.891680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81ca22-ce8a-42bb-84ef-bfced48db7e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:03.891953 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81ca22-ce8a-42bb-84ef-bfced48db7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:03.892219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81ca22-ce8a-42bb-84ef-bfced48db7e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:03.892895 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81ca22-ce8a-42bb-84ef-bfced48db7e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:03.894882 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81ca22-ce8a-42bb-84ef-bfced48db7e8 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:03.920504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 931/1883] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:13:03 2022] POST /identity/v3/users => generated 296 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:03.929677 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81ca22-ce8a-42bb-84ef-bfced48db7e8 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:03.932351 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-ed81ca22-ce8a-42bb-84ef-bfced48db7e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:03.935482 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b7137-286a-4e1a-a37e-670d7f9e2059 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:03.940231 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6782bc-3dc8-4050-bebc-74ed78104f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:03.941465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6782bc-3dc8-4050-bebc-74ed78104f5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:03.941465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6782bc-3dc8-4050-bebc-74ed78104f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:03.941465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6782bc-3dc8-4050-bebc-74ed78104f5e admin admin] PATH_INFO: `/v3/users/4061c8c2422a4fd68342f1fe6524f421` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:03.942582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6782bc-3dc8-4050-bebc-74ed78104f5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4061c8c2422a4fd68342f1fe6524f421)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:03.944566 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6782bc-3dc8-4050-bebc-74ed78104f5e admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:03.944876 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81ca22-ce8a-42bb-84ef-bfced48db7e8 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:03.946344 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:03.950419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9aa6aa-373c-41e6-8f5b-e9c8d854b227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:03.951297 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9aa6aa-373c-41e6-8f5b-e9c8d854b227 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:03.951475 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9aa6aa-373c-41e6-8f5b-e9c8d854b227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:03.951755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9aa6aa-373c-41e6-8f5b-e9c8d854b227 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:03.952508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9aa6aa-373c-41e6-8f5b-e9c8d854b227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:03.954334 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9aa6aa-373c-41e6-8f5b-e9c8d854b227 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:03.966450 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 932/1884] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:13:03.970748 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9aa6aa-373c-41e6-8f5b-e9c8d854b227 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:03.983282 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9aa6aa-373c-41e6-8f5b-e9c8d854b227 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:03.993227 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd47f9-7d2d-4881-b338-5eef4c466cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:03.994278 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd47f9-7d2d-4881-b338-5eef4c466cff admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:03.994675 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd47f9-7d2d-4881-b338-5eef4c466cff admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:03.994823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd47f9-7d2d-4881-b338-5eef4c466cff admin admin] PATH_INFO: `/v3/projects/85d31bff0d5944c9ab94ce6a1adba203/users/e2c53329df3748548a20afc3bdcadf4b/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:04.010425 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-4a6782bc-3dc8-4050-bebc-74ed78104f5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4061c8c2422a4fd68342f1fe6524f421'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:04.012872 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd47f9-7d2d-4881-b338-5eef4c466cff admin admin] RBAC: Authorizing `identity:create_grant(project_id=85d31bff0d5944c9ab94ce6a1adba203, user_id=e2c53329df3748548a20afc3bdcadf4b, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:04.016513 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd47f9-7d2d-4881-b338-5eef4c466cff admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:04.020157 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:04.021036 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-4a6782bc-3dc8-4050-bebc-74ed78104f5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4061c8c2422a4fd68342f1fe6524f421'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:04.038721 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-4a6782bc-3dc8-4050-bebc-74ed78104f5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4061c8c2422a4fd68342f1fe6524f421'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:04.039522 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-4a6782bc-3dc8-4050-bebc-74ed78104f5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4061c8c2422a4fd68342f1fe6524f421'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:04.040241 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 933/1885] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:03 2022] PUT /identity/v3/projects/85d31bff0d5944c9ab94ce6a1adba203/users/e2c53329df3748548a20afc3bdcadf4b/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:04.045922 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd47f9-7d2d-4881-b338-5eef4c466cff admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:04.048821 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-19cd47f9-7d2d-4881-b338-5eef4c466cff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:04.050334 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-4a6782bc-3dc8-4050-bebc-74ed78104f5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4061c8c2422a4fd68342f1fe6524f421'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:04.064913 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 953/1886] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:03 2022] DELETE /identity/v3/users/4061c8c2422a4fd68342f1fe6524f421 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:04.144999 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd47f9-7d2d-4881-b338-5eef4c466cff admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:04.256744 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e54ab-28ca-4ac2-b978-a8974e291d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:04.257719 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e54ab-28ca-4ac2-b978-a8974e291d62 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:04.258046 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e54ab-28ca-4ac2-b978-a8974e291d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:04.258331 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e54ab-28ca-4ac2-b978-a8974e291d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:04.259662 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e54ab-28ca-4ac2-b978-a8974e291d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:04.261495 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e54ab-28ca-4ac2-b978-a8974e291d62 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:04.276082 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 934/1887] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:13:04.280595 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6782bc-3dc8-4050-bebc-74ed78104f5e admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:04.301449 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6782bc-3dc8-4050-bebc-74ed78104f5e admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:04.314971 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd9bd4-f2ba-4011-85d5-65588b0fb6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:04.314971 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd9bd4-f2ba-4011-85d5-65588b0fb6dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:04.314971 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd9bd4-f2ba-4011-85d5-65588b0fb6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:04.314971 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd9bd4-f2ba-4011-85d5-65588b0fb6dc admin admin] PATH_INFO: `/v3/projects/85d31bff0d5944c9ab94ce6a1adba203/users/e2c53329df3748548a20afc3bdcadf4b/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:04.315357 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd9bd4-f2ba-4011-85d5-65588b0fb6dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=85d31bff0d5944c9ab94ce6a1adba203, user_id=e2c53329df3748548a20afc3bdcadf4b, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:04.315357 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd9bd4-f2ba-4011-85d5-65588b0fb6dc admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:04.336561 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 954/1888] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:04 2022] PUT /identity/v3/projects/85d31bff0d5944c9ab94ce6a1adba203/users/e2c53329df3748548a20afc3bdcadf4b/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:04.349713 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ca386-b6ac-4da9-934d-c6825b167ebc None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:04.350174 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ca386-b6ac-4da9-934d-c6825b167ebc None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:04.350989 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ca386-b6ac-4da9-934d-c6825b167ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:04.397884 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-f47ca386-b6ac-4da9-934d-c6825b167ebc None None] MFA Rules not processed for user `e2c53329df3748548a20afc3bdcadf4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:04.399552 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f47ca386-b6ac-4da9-934d-c6825b167ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:04.519721 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd9bd4-f2ba-4011-85d5-65588b0fb6dc admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:04.521779 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-f9cd9bd4-f2ba-4011-85d5-65588b0fb6dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:04.546551 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 935/1889] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:04 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:04.580769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ca386-b6ac-4da9-934d-c6825b167ebc None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:04.588419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f47ca386-b6ac-4da9-934d-c6825b167ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:04.634546 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:04.637430 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd9bd4-f2ba-4011-85d5-65588b0fb6dc admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:04.652102 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d95bad6-74c4-44b6-9815-c33e4342fbf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:04.652759 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95bad6-74c4-44b6-9815-c33e4342fbf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:04.653031 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95bad6-74c4-44b6-9815-c33e4342fbf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:04.653328 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95bad6-74c4-44b6-9815-c33e4342fbf0 admin admin] PATH_INFO: `/v3/projects/d61da770f51a492ea01d2aad14a35147` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:04.654532 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d95bad6-74c4-44b6-9815-c33e4342fbf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d61da770f51a492ea01d2aad14a35147)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:04.656384 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d95bad6-74c4-44b6-9815-c33e4342fbf0 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:04.676262 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ca386-b6ac-4da9-934d-c6825b167ebc None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:04.681206 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd82d5-6e57-40a7-9999-9c7a43dc4529 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:04.681836 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd82d5-6e57-40a7-9999-9c7a43dc4529 service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:04.682344 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd82d5-6e57-40a7-9999-9c7a43dc4529 service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:04.682492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd82d5-6e57-40a7-9999-9c7a43dc4529 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:04.693032 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd82d5-6e57-40a7-9999-9c7a43dc4529 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:04.694780 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd82d5-6e57-40a7-9999-9c7a43dc4529 service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:04.708760 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 936/1890] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:04 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:04.713892 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd82d5-6e57-40a7-9999-9c7a43dc4529 service cinder] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:04.715883 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-efbd82d5-6e57-40a7-9999-9c7a43dc4529 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:04.730711 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd82d5-6e57-40a7-9999-9c7a43dc4529 service cinder] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:04.744651 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2d95bad6-74c4-44b6-9815-c33e4342fbf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd61da770f51a492ea01d2aad14a35147', 'request_id': 'req-2d95bad6-74c4-44b6-9815-c33e4342fbf0'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:04.754084 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2d95bad6-74c4-44b6-9815-c33e4342fbf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd61da770f51a492ea01d2aad14a35147', 'request_id': 'req-2d95bad6-74c4-44b6-9815-c33e4342fbf0'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:04.774187 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2d95bad6-74c4-44b6-9815-c33e4342fbf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd61da770f51a492ea01d2aad14a35147', 'request_id': 'req-2d95bad6-74c4-44b6-9815-c33e4342fbf0'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:04.779411 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:04.797076 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2d95bad6-74c4-44b6-9815-c33e4342fbf0 admin admin] The token cache is being invalidate because project d61da770f51a492ea01d2aad14a35147 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:04.797787 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2d95bad6-74c4-44b6-9815-c33e4342fbf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:04.798980 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2d95bad6-74c4-44b6-9815-c33e4342fbf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:04.801006 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ea1bf-619b-408d-980d-a4a2c898fe83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:04.801565 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 955/1891] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:04 2022] DELETE /identity/v3/projects/d61da770f51a492ea01d2aad14a35147 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:13:04.805328 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d95bad6-74c4-44b6-9815-c33e4342fbf0 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:04.816270 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-2d95bad6-74c4-44b6-9815-c33e4342fbf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:04.816482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ea1bf-619b-408d-980d-a4a2c898fe83 service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:04.816482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ea1bf-619b-408d-980d-a4a2c898fe83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:04.816482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ea1bf-619b-408d-980d-a4a2c898fe83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:04.818513 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f89ea1bf-619b-408d-980d-a4a2c898fe83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:04.904652 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d95bad6-74c4-44b6-9815-c33e4342fbf0 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:04.912680 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef74d3c-4336-489d-9709-1997317807cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:04.913897 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef74d3c-4336-489d-9709-1997317807cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:04.914402 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef74d3c-4336-489d-9709-1997317807cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:04.914823 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef74d3c-4336-489d-9709-1997317807cf admin admin] PATH_INFO: `/v3/users/ac8d8af2e45e45eab8180c39daca6535` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:04.916841 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef74d3c-4336-489d-9709-1997317807cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac8d8af2e45e45eab8180c39daca6535)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:04.919404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef74d3c-4336-489d-9709-1997317807cf admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:04.923824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89ea1bf-619b-408d-980d-a4a2c898fe83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:04.925369 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89ea1bf-619b-408d-980d-a4a2c898fe83 service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:04.935912 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 937/1892] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:04 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:05.004961 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-cef74d3c-4336-489d-9709-1997317807cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac8d8af2e45e45eab8180c39daca6535'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:05.020164 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-cef74d3c-4336-489d-9709-1997317807cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac8d8af2e45e45eab8180c39daca6535'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:05.052602 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-cef74d3c-4336-489d-9709-1997317807cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac8d8af2e45e45eab8180c39daca6535'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:05.053863 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-cef74d3c-4336-489d-9709-1997317807cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac8d8af2e45e45eab8180c39daca6535'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:05.070458 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-cef74d3c-4336-489d-9709-1997317807cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac8d8af2e45e45eab8180c39daca6535'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:05.090828 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 956/1893] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:04 2022] DELETE /identity/v3/users/ac8d8af2e45e45eab8180c39daca6535 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:05.218224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ea1bf-619b-408d-980d-a4a2c898fe83 service cinder] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:05.220067 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f89ea1bf-619b-408d-980d-a4a2c898fe83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:05.307166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ea1bf-619b-408d-980d-a4a2c898fe83 service cinder] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:05.328389 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:05.371411 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f3680-fe39-465f-b6c8-930699cb7e31 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2c53329df3748548a20afc3bdcadf4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85d31bff0d5944c9ab94ce6a1adba203', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:05.372132 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f3680-fe39-465f-b6c8-930699cb7e31 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:05.372504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f3680-fe39-465f-b6c8-930699cb7e31 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:05.372853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f3680-fe39-465f-b6c8-930699cb7e31 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:05.373568 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f3680-fe39-465f-b6c8-930699cb7e31 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:13:05.373926 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f3680-fe39-465f-b6c8-930699cb7e31 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:05.375597 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f3680-fe39-465f-b6c8-930699cb7e31 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:05.388802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 938/1894] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:13:05.393393 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef74d3c-4336-489d-9709-1997317807cf admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:05.394898 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-cef74d3c-4336-489d-9709-1997317807cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:05.410025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef74d3c-4336-489d-9709-1997317807cf admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:05.415488 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb634a50-17cf-44ea-9740-8723f91ad15a tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2c53329df3748548a20afc3bdcadf4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85d31bff0d5944c9ab94ce6a1adba203', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:05.416118 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb634a50-17cf-44ea-9740-8723f91ad15a tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:05.416396 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb634a50-17cf-44ea-9740-8723f91ad15a tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:05.416660 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb634a50-17cf-44ea-9740-8723f91ad15a tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:05.417357 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb634a50-17cf-44ea-9740-8723f91ad15a tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:05.419295 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb634a50-17cf-44ea-9740-8723f91ad15a tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:05.450155 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 957/1895] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:13:05 2022] POST /identity/v3/projects => generated 354 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:05.455221 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f3680-fe39-465f-b6c8-930699cb7e31 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:05.471630 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f3680-fe39-465f-b6c8-930699cb7e31 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:05.509906 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb634a50-17cf-44ea-9740-8723f91ad15a tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:05.511980 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-cb634a50-17cf-44ea-9740-8723f91ad15a tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:05.573359 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237fcda1-9e8b-4b92-b28a-8a03dd8ee597 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2c53329df3748548a20afc3bdcadf4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85d31bff0d5944c9ab94ce6a1adba203', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:05.573359 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237fcda1-9e8b-4b92-b28a-8a03dd8ee597 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:05.573359 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237fcda1-9e8b-4b92-b28a-8a03dd8ee597 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:05.573639 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237fcda1-9e8b-4b92-b28a-8a03dd8ee597 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:05.573639 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237fcda1-9e8b-4b92-b28a-8a03dd8ee597 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:13:05.573639 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237fcda1-9e8b-4b92-b28a-8a03dd8ee597 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:05.573639 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237fcda1-9e8b-4b92-b28a-8a03dd8ee597 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:05.583910 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 939/1896] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:13:05.604255 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:05.620351 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb634a50-17cf-44ea-9740-8723f91ad15a tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:05.625375 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae1fa8-9198-4c3c-a1f4-9b0c238f6c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:05.626030 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae1fa8-9198-4c3c-a1f4-9b0c238f6c33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:05.626591 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae1fa8-9198-4c3c-a1f4-9b0c238f6c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:05.626863 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae1fa8-9198-4c3c-a1f4-9b0c238f6c33 admin admin] PATH_INFO: `/v3/projects/b6f10386e00b49e28c8bdf1ed9b9b710` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:05.627449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237fcda1-9e8b-4b92-b28a-8a03dd8ee597 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:05.628014 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae1fa8-9198-4c3c-a1f4-9b0c238f6c33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6f10386e00b49e28c8bdf1ed9b9b710)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:05.630044 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae1fa8-9198-4c3c-a1f4-9b0c238f6c33 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:05.637881 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237fcda1-9e8b-4b92-b28a-8a03dd8ee597 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:05.642876 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f44e1-89cb-4dda-8ef3-65d6fac90344 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2c53329df3748548a20afc3bdcadf4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '85d31bff0d5944c9ab94ce6a1adba203', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:05.643516 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f44e1-89cb-4dda-8ef3-65d6fac90344 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:05.643791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f44e1-89cb-4dda-8ef3-65d6fac90344 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:05.644071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f44e1-89cb-4dda-8ef3-65d6fac90344 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] PATH_INFO: `/v3/projects/4e370b346e7540bca52b37ef0f874c80` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:05.645259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9f44e1-89cb-4dda-8ef3-65d6fac90344 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4e370b346e7540bca52b37ef0f874c80)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:05.647180 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9f44e1-89cb-4dda-8ef3-65d6fac90344 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:05.682945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-c1ae1fa8-9198-4c3c-a1f4-9b0c238f6c33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6f10386e00b49e28c8bdf1ed9b9b710', 'request_id': 'req-c1ae1fa8-9198-4c3c-a1f4-9b0c238f6c33'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:05.688862 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-c1ae1fa8-9198-4c3c-a1f4-9b0c238f6c33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6f10386e00b49e28c8bdf1ed9b9b710', 'request_id': 'req-c1ae1fa8-9198-4c3c-a1f4-9b0c238f6c33'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:05.695293 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-c1ae1fa8-9198-4c3c-a1f4-9b0c238f6c33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6f10386e00b49e28c8bdf1ed9b9b710', 'request_id': 'req-c1ae1fa8-9198-4c3c-a1f4-9b0c238f6c33'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:05.700164 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-6f9f44e1-89cb-4dda-8ef3-65d6fac90344 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e370b346e7540bca52b37ef0f874c80', 'request_id': 'req-6f9f44e1-89cb-4dda-8ef3-65d6fac90344'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:05.708560 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-6f9f44e1-89cb-4dda-8ef3-65d6fac90344 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e370b346e7540bca52b37ef0f874c80', 'request_id': 'req-6f9f44e1-89cb-4dda-8ef3-65d6fac90344'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:05.713120 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-c1ae1fa8-9198-4c3c-a1f4-9b0c238f6c33 admin admin] The token cache is being invalidate because project b6f10386e00b49e28c8bdf1ed9b9b710 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:05.713433 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-c1ae1fa8-9198-4c3c-a1f4-9b0c238f6c33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:05.714181 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-c1ae1fa8-9198-4c3c-a1f4-9b0c238f6c33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:05.714951 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-6f9f44e1-89cb-4dda-8ef3-65d6fac90344 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e370b346e7540bca52b37ef0f874c80', 'request_id': 'req-6f9f44e1-89cb-4dda-8ef3-65d6fac90344'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:05.716122 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 958/1897] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:05 2022] DELETE /identity/v3/projects/b6f10386e00b49e28c8bdf1ed9b9b710 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:05.721752 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54aa6d-8696-4d8f-86c9-2598f020bb20 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:05.722033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54aa6d-8696-4d8f-86c9-2598f020bb20 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:05.722305 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54aa6d-8696-4d8f-86c9-2598f020bb20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:05.727532 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-6f9f44e1-89cb-4dda-8ef3-65d6fac90344 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] The token cache is being invalidate because project 4e370b346e7540bca52b37ef0f874c80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:05.727871 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-6f9f44e1-89cb-4dda-8ef3-65d6fac90344 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:05.728572 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-6f9f44e1-89cb-4dda-8ef3-65d6fac90344 tempest-VolumeQuotaClassesTest-106936370 tempest-VolumeQuotaClassesTest-106936370-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:05.730870 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 940/1898] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:05 2022] DELETE /identity/v3/projects/4e370b346e7540bca52b37ef0f874c80 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:13:05.743724 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-9a54aa6d-8696-4d8f-86c9-2598f020bb20 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:05.745018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-9a54aa6d-8696-4d8f-86c9-2598f020bb20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:05.814103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 959/1899] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:13:05.823351 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839c407-1e1e-40f6-9801-dbe6a5b47269 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:05.823641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839c407-1e1e-40f6-9801-dbe6a5b47269 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:05.823931 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839c407-1e1e-40f6-9801-dbe6a5b47269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:05.866311 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-a839c407-1e1e-40f6-9801-dbe6a5b47269 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:05.867866 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-a839c407-1e1e-40f6-9801-dbe6a5b47269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:05.881914 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 941/1900] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:13:05.886240 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54aa6d-8696-4d8f-86c9-2598f020bb20 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:05.902884 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54aa6d-8696-4d8f-86c9-2598f020bb20 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:05.916041 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a55259-7081-4b27-9a2f-35d4a5776a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:05.916767 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a55259-7081-4b27-9a2f-35d4a5776a39 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:05.917112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a55259-7081-4b27-9a2f-35d4a5776a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:05.917849 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a55259-7081-4b27-9a2f-35d4a5776a39 admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:05.918112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a55259-7081-4b27-9a2f-35d4a5776a39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:13:05.918444 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a55259-7081-4b27-9a2f-35d4a5776a39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:05.920138 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a55259-7081-4b27-9a2f-35d4a5776a39 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:05.966676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839c407-1e1e-40f6-9801-dbe6a5b47269 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:05.966930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 960/1901] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:13:06.002894 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839c407-1e1e-40f6-9801-dbe6a5b47269 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:06.025454 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c5e73-b3ab-4cda-9f8c-342687583da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:06.026546 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c5e73-b3ab-4cda-9f8c-342687583da0 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:06.035284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c5e73-b3ab-4cda-9f8c-342687583da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:06.036414 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c5e73-b3ab-4cda-9f8c-342687583da0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:06.037696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c5e73-b3ab-4cda-9f8c-342687583da0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:06.043563 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c5e73-b3ab-4cda-9f8c-342687583da0 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:06.154297 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a55259-7081-4b27-9a2f-35d4a5776a39 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:06.160198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 942/1902] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:13:05 2022] POST /identity/v3/projects => generated 377 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:06.168806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a55259-7081-4b27-9a2f-35d4a5776a39 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:06.214923 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c5e73-b3ab-4cda-9f8c-342687583da0 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:06.217954 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-a45c5e73-b3ab-4cda-9f8c-342687583da0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:06.232325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:06.320693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b5a9e-6db5-4970-9932-98ff4283b431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:06.322509 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b5a9e-6db5-4970-9932-98ff4283b431 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:06.323765 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b5a9e-6db5-4970-9932-98ff4283b431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:06.323765 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b5a9e-6db5-4970-9932-98ff4283b431 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:06.325556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b5a9e-6db5-4970-9932-98ff4283b431 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:06.328113 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b5a9e-6db5-4970-9932-98ff4283b431 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:06.357453 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:06.367788 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 961/1903] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:13:06 2022] POST /identity/v3/users => generated 289 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:06.373534 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b5a9e-6db5-4970-9932-98ff4283b431 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:06.378513 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c5e73-b3ab-4cda-9f8c-342687583da0 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:06.385945 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5ad83-a438-4c02-a146-7183f4e2c007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:06.386778 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5ad83-a438-4c02-a146-7183f4e2c007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:06.387130 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5ad83-a438-4c02-a146-7183f4e2c007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:06.387462 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5ad83-a438-4c02-a146-7183f4e2c007 admin admin] PATH_INFO: `/v3/users/7ef1b695e6fd4f98a98464512cc042d5` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:06.387859 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b5a9e-6db5-4970-9932-98ff4283b431 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:06.388753 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be5ad83-a438-4c02-a146-7183f4e2c007 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ef1b695e6fd4f98a98464512cc042d5)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:06.391489 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be5ad83-a438-4c02-a146-7183f4e2c007 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:06.395451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7bdb7b-8b0d-40f1-956c-33b5ab73c3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:06.396741 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bdb7b-8b0d-40f1-956c-33b5ab73c3f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:06.397779 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bdb7b-8b0d-40f1-956c-33b5ab73c3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:06.398254 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bdb7b-8b0d-40f1-956c-33b5ab73c3f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:06.399588 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7bdb7b-8b0d-40f1-956c-33b5ab73c3f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:06.402489 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7bdb7b-8b0d-40f1-956c-33b5ab73c3f8 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:06.428552 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 962/1904] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:06.432864 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7bdb7b-8b0d-40f1-956c-33b5ab73c3f8 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:06.448936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7bdb7b-8b0d-40f1-956c-33b5ab73c3f8 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:06.456084 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7cff5-4f11-4662-9f72-31464140ed10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:06.457911 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7cff5-4f11-4662-9f72-31464140ed10 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:06.458409 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7cff5-4f11-4662-9f72-31464140ed10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:06.459017 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7cff5-4f11-4662-9f72-31464140ed10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:06.459999 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7cff5-4f11-4662-9f72-31464140ed10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:06.462012 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7cff5-4f11-4662-9f72-31464140ed10 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:06.470145 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-4be5ad83-a438-4c02-a146-7183f4e2c007 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ef1b695e6fd4f98a98464512cc042d5'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:06.477414 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 963/1905] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:13:06.483028 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7cff5-4f11-4662-9f72-31464140ed10 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:06.489637 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-4be5ad83-a438-4c02-a146-7183f4e2c007 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ef1b695e6fd4f98a98464512cc042d5'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:06.502372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7cff5-4f11-4662-9f72-31464140ed10 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:06.512095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a5d37-6fd0-4bbb-b162-2dd232a0ac54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:06.513320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a5d37-6fd0-4bbb-b162-2dd232a0ac54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:06.513768 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a5d37-6fd0-4bbb-b162-2dd232a0ac54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:06.514224 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a5d37-6fd0-4bbb-b162-2dd232a0ac54 admin admin] PATH_INFO: `/v3/projects/005a5a842d0f4f4ba39af249fa0f9121/users/5649a36703744e28bfbbdc18331c16fa/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:06.535323 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a5d37-6fd0-4bbb-b162-2dd232a0ac54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=005a5a842d0f4f4ba39af249fa0f9121, user_id=5649a36703744e28bfbbdc18331c16fa, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:06.537921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-4be5ad83-a438-4c02-a146-7183f4e2c007 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ef1b695e6fd4f98a98464512cc042d5'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:06.539068 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-4be5ad83-a438-4c02-a146-7183f4e2c007 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ef1b695e6fd4f98a98464512cc042d5'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:06.539359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a5d37-6fd0-4bbb-b162-2dd232a0ac54 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:06.559251 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-4be5ad83-a438-4c02-a146-7183f4e2c007 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ef1b695e6fd4f98a98464512cc042d5'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:06.560182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 964/1906] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:06 2022] PUT /identity/v3/projects/005a5a842d0f4f4ba39af249fa0f9121/users/5649a36703744e28bfbbdc18331c16fa/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:13:06.573653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373f9f1-8bd4-4f49-a3b0-c4486ce5e572 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:06.575104 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373f9f1-8bd4-4f49-a3b0-c4486ce5e572 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:06.576184 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373f9f1-8bd4-4f49-a3b0-c4486ce5e572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:06.581640 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 943/1907] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:06 2022] DELETE /identity/v3/users/7ef1b695e6fd4f98a98464512cc042d5 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:06.586225 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889f8ee-00ae-4659-8e2b-9d0c96785388 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:06.586576 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889f8ee-00ae-4659-8e2b-9d0c96785388 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:06.586982 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889f8ee-00ae-4659-8e2b-9d0c96785388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:06.650055 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-b373f9f1-8bd4-4f49-a3b0-c4486ce5e572 None None] MFA Rules not processed for user `5649a36703744e28bfbbdc18331c16fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:06.651315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-b373f9f1-8bd4-4f49-a3b0-c4486ce5e572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:06.659659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-f889f8ee-00ae-4659-8e2b-9d0c96785388 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:06.671979 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f889f8ee-00ae-4659-8e2b-9d0c96785388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:06.839875 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 944/1908] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:13:06.854699 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f889f8ee-00ae-4659-8e2b-9d0c96785388 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:06.856538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f889f8ee-00ae-4659-8e2b-9d0c96785388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:06.863084 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 965/1909] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:13:06.864445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373f9f1-8bd4-4f49-a3b0-c4486ce5e572 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:06.866438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-b373f9f1-8bd4-4f49-a3b0-c4486ce5e572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:06.913394 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:06.991982 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f889f8ee-00ae-4659-8e2b-9d0c96785388 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:07.000096 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bbb9db-c743-402d-aa46-3409b1b7dc87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:07.000542 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbb9db-c743-402d-aa46-3409b1b7dc87 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:07.001269 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbb9db-c743-402d-aa46-3409b1b7dc87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:07.001544 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbb9db-c743-402d-aa46-3409b1b7dc87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:07.020138 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bbb9db-c743-402d-aa46-3409b1b7dc87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:07.021907 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bbb9db-c743-402d-aa46-3409b1b7dc87 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:07.036566 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373f9f1-8bd4-4f49-a3b0-c4486ce5e572 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:07.037773 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 945/1910] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:07.053127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d2c98-44a0-411e-a07c-d9da7db69651 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fe5293447346eaafd16818e5bd12f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:07.054910 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d2c98-44a0-411e-a07c-d9da7db69651 service glance] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:07.055299 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d2c98-44a0-411e-a07c-d9da7db69651 service glance] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:07.056203 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d2c98-44a0-411e-a07c-d9da7db69651 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:07.071013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d2c98-44a0-411e-a07c-d9da7db69651 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:07.072700 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d2c98-44a0-411e-a07c-d9da7db69651 service glance] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:07.088980 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 966/1911] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:07.569939 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bbb9db-c743-402d-aa46-3409b1b7dc87 service neutron] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:07.573635 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f4bbb9db-c743-402d-aa46-3409b1b7dc87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:07.611843 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bbb9db-c743-402d-aa46-3409b1b7dc87 service neutron] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:07.622432 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5107025-ce31-4611-8364-18d17e396468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:07.624233 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5107025-ce31-4611-8364-18d17e396468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:07.625780 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5107025-ce31-4611-8364-18d17e396468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:07.627128 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5107025-ce31-4611-8364-18d17e396468 admin admin] PATH_INFO: `/v3/projects/99c1fddc22af4a2c912110853edd6316` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:07.629446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5107025-ce31-4611-8364-18d17e396468 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99c1fddc22af4a2c912110853edd6316)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:07.633508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5107025-ce31-4611-8364-18d17e396468 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:07.725779 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d5107025-ce31-4611-8364-18d17e396468 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99c1fddc22af4a2c912110853edd6316', 'request_id': 'req-d5107025-ce31-4611-8364-18d17e396468'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:07.741616 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d5107025-ce31-4611-8364-18d17e396468 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99c1fddc22af4a2c912110853edd6316', 'request_id': 'req-d5107025-ce31-4611-8364-18d17e396468'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:07.770853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d5107025-ce31-4611-8364-18d17e396468 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99c1fddc22af4a2c912110853edd6316', 'request_id': 'req-d5107025-ce31-4611-8364-18d17e396468'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:07.788993 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d5107025-ce31-4611-8364-18d17e396468 admin admin] The token cache is being invalidate because project 99c1fddc22af4a2c912110853edd6316 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:07.790565 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d5107025-ce31-4611-8364-18d17e396468 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:07.791757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d5107025-ce31-4611-8364-18d17e396468 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:07.794083 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 946/1912] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:07 2022] DELETE /identity/v3/projects/99c1fddc22af4a2c912110853edd6316 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:07.798241 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d2c98-44a0-411e-a07c-d9da7db69651 service glance] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:07.816340 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d2c98-44a0-411e-a07c-d9da7db69651 service glance] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:07.874067 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:07.933634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3689c10-60a0-4767-9080-87a30bc0836c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:07.934396 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3689c10-60a0-4767-9080-87a30bc0836c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:07.934749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3689c10-60a0-4767-9080-87a30bc0836c admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:07.935088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3689c10-60a0-4767-9080-87a30bc0836c admin admin] PATH_INFO: `/v3/users/e2c53329df3748548a20afc3bdcadf4b` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:07.936397 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3689c10-60a0-4767-9080-87a30bc0836c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2c53329df3748548a20afc3bdcadf4b)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:07.938477 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3689c10-60a0-4767-9080-87a30bc0836c admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:08.016582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e3689c10-60a0-4767-9080-87a30bc0836c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2c53329df3748548a20afc3bdcadf4b'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:08.025739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e3689c10-60a0-4767-9080-87a30bc0836c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2c53329df3748548a20afc3bdcadf4b'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:08.040472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e3689c10-60a0-4767-9080-87a30bc0836c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2c53329df3748548a20afc3bdcadf4b'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:08.041319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e3689c10-60a0-4767-9080-87a30bc0836c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2c53329df3748548a20afc3bdcadf4b'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:08.048954 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e3689c10-60a0-4767-9080-87a30bc0836c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2c53329df3748548a20afc3bdcadf4b'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:08.055520 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 967/1913] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:07 2022] DELETE /identity/v3/users/e2c53329df3748548a20afc3bdcadf4b => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:09.567274 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efa079-f741-4910-aea9-706702eeab1e None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:09.568020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efa079-f741-4910-aea9-706702eeab1e None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:09.568697 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efa079-f741-4910-aea9-706702eeab1e None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:09.573164 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 947/1914] 10.222.0.44 () {54 vars in 921 bytes} [Sun Sep 11 15:13:09 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:13:09.585820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205836c4-836c-4b01-8163-d03195c7bf89 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:09.586811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205836c4-836c-4b01-8163-d03195c7bf89 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:09.587409 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205836c4-836c-4b01-8163-d03195c7bf89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:09.633059 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-205836c4-836c-4b01-8163-d03195c7bf89 None None] MFA Rules not processed for user `453297dac64046d19fc1c331d2d85abc`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:09.640332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-205836c4-836c-4b01-8163-d03195c7bf89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:09.891984 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 968/1915] 10.222.0.44 () {58 vars in 1018 bytes} [Sun Sep 11 15:13:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:09.919499 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10efa079-f741-4910-aea9-706702eeab1e None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:09.922066 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-10efa079-f741-4910-aea9-706702eeab1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:09.965791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10efa079-f741-4910-aea9-706702eeab1e None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:09.974336 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe5d62-8670-43bf-b03d-bba725dbaa30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:09.975328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe5d62-8670-43bf-b03d-bba725dbaa30 service nova] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:09.975606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe5d62-8670-43bf-b03d-bba725dbaa30 service nova] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:09.975879 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe5d62-8670-43bf-b03d-bba725dbaa30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:09.992906 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfe5d62-8670-43bf-b03d-bba725dbaa30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:09.996862 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfe5d62-8670-43bf-b03d-bba725dbaa30 service nova] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:10.023214 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 948/1916] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:13:10.060254 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205836c4-836c-4b01-8163-d03195c7bf89 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:10.061922 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-205836c4-836c-4b01-8163-d03195c7bf89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:10.121800 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe5d62-8670-43bf-b03d-bba725dbaa30 service nova] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:10.122662 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-1bfe5d62-8670-43bf-b03d-bba725dbaa30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:10.199162 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:10.262408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205836c4-836c-4b01-8163-d03195c7bf89 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:10.268016 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe5d62-8670-43bf-b03d-bba725dbaa30 service nova] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:10.272792 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354b593-fe5d-4029-8877-e9b612132e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:10.278246 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354b593-fe5d-4029-8877-e9b612132e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:10.278956 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354b593-fe5d-4029-8877-e9b612132e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:10.279361 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354b593-fe5d-4029-8877-e9b612132e53 admin admin] PATH_INFO: `/v3/projects/85d31bff0d5944c9ab94ce6a1adba203` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:10.280864 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9354b593-fe5d-4029-8877-e9b612132e53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85d31bff0d5944c9ab94ce6a1adba203)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:10.282773 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7af58-49ac-4bc1-a06c-6069ec654afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:10.283683 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7af58-49ac-4bc1-a06c-6069ec654afe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:10.283868 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9354b593-fe5d-4029-8877-e9b612132e53 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:10.284432 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7af58-49ac-4bc1-a06c-6069ec654afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:10.284888 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7af58-49ac-4bc1-a06c-6069ec654afe admin admin] PATH_INFO: `/v3/users/0d326898402547a5beca3aaf9b5c2c62` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:10.286691 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb7af58-49ac-4bc1-a06c-6069ec654afe admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d326898402547a5beca3aaf9b5c2c62)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:10.289002 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb7af58-49ac-4bc1-a06c-6069ec654afe admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:10.369567 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-9354b593-fe5d-4029-8877-e9b612132e53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85d31bff0d5944c9ab94ce6a1adba203', 'request_id': 'req-9354b593-fe5d-4029-8877-e9b612132e53'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:10.373350 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:10.376193 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-9354b593-fe5d-4029-8877-e9b612132e53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85d31bff0d5944c9ab94ce6a1adba203', 'request_id': 'req-9354b593-fe5d-4029-8877-e9b612132e53'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:10.384484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-9354b593-fe5d-4029-8877-e9b612132e53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85d31bff0d5944c9ab94ce6a1adba203', 'request_id': 'req-9354b593-fe5d-4029-8877-e9b612132e53'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:10.385474 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-5cb7af58-49ac-4bc1-a06c-6069ec654afe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d326898402547a5beca3aaf9b5c2c62'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:10.398409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-5cb7af58-49ac-4bc1-a06c-6069ec654afe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d326898402547a5beca3aaf9b5c2c62'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:10.401265 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-9354b593-fe5d-4029-8877-e9b612132e53 admin admin] The token cache is being invalidate because project 85d31bff0d5944c9ab94ce6a1adba203 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:10.401661 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-9354b593-fe5d-4029-8877-e9b612132e53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:10.402496 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-9354b593-fe5d-4029-8877-e9b612132e53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:10.404425 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 969/1917] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:10 2022] DELETE /identity/v3/projects/85d31bff0d5944c9ab94ce6a1adba203 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:10.409848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf115f5-6ce6-4477-ac32-190fb74e1c1e None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:10.410288 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf115f5-6ce6-4477-ac32-190fb74e1c1e None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:10.410749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf115f5-6ce6-4477-ac32-190fb74e1c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:10.421830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-5cb7af58-49ac-4bc1-a06c-6069ec654afe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d326898402547a5beca3aaf9b5c2c62'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:10.422727 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-5cb7af58-49ac-4bc1-a06c-6069ec654afe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d326898402547a5beca3aaf9b5c2c62'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:10.431379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-5cb7af58-49ac-4bc1-a06c-6069ec654afe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d326898402547a5beca3aaf9b5c2c62'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:10.443302 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 949/1918] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:10 2022] DELETE /identity/v3/users/0d326898402547a5beca3aaf9b5c2c62 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:13:10.450712 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a944a4-ebea-416f-9f76-18973b7c54ee None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:10.451049 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a944a4-ebea-416f-9f76-18973b7c54ee None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:10.451265 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a944a4-ebea-416f-9f76-18973b7c54ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:10.458891 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-ebf115f5-6ce6-4477-ac32-190fb74e1c1e None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:10.461050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-ebf115f5-6ce6-4477-ac32-190fb74e1c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:10.484990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-88a944a4-ebea-416f-9f76-18973b7c54ee None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:10.486568 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-88a944a4-ebea-416f-9f76-18973b7c54ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:10.566727 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 950/1919] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:10.577432 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a944a4-ebea-416f-9f76-18973b7c54ee None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:10.579665 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-88a944a4-ebea-416f-9f76-18973b7c54ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:10.608810 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 970/1920] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:13:10.610289 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d685d-c0a8-407a-b68f-f958a2fd03e3 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:10.610577 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d685d-c0a8-407a-b68f-f958a2fd03e3 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:10.610851 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d685d-c0a8-407a-b68f-f958a2fd03e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:10.637685 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-128d685d-c0a8-407a-b68f-f958a2fd03e3 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:10.639248 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-128d685d-c0a8-407a-b68f-f958a2fd03e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:10.651406 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 971/1921] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:10.652362 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d685d-c0a8-407a-b68f-f958a2fd03e3 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:10.668264 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d685d-c0a8-407a-b68f-f958a2fd03e3 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:10.671605 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a944a4-ebea-416f-9f76-18973b7c54ee None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:10.680258 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea506f3-1ab7-420f-b4a3-8be9eca1f18e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:10.681199 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea506f3-1ab7-420f-b4a3-8be9eca1f18e service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:10.682003 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea506f3-1ab7-420f-b4a3-8be9eca1f18e service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:10.682421 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea506f3-1ab7-420f-b4a3-8be9eca1f18e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:10.683665 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c34a5-f216-48da-9e3c-3afeaad6d6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:10.684595 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c34a5-f216-48da-9e3c-3afeaad6d6e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:10.685008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c34a5-f216-48da-9e3c-3afeaad6d6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:10.685298 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c34a5-f216-48da-9e3c-3afeaad6d6e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:10.686045 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c34a5-f216-48da-9e3c-3afeaad6d6e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:13:10.686410 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c34a5-f216-48da-9e3c-3afeaad6d6e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:10.688078 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c34a5-f216-48da-9e3c-3afeaad6d6e9 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:10.692697 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea506f3-1ab7-420f-b4a3-8be9eca1f18e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:10.694494 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea506f3-1ab7-420f-b4a3-8be9eca1f18e service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:10.713615 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 972/1922] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:13:10.715263 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 951/1923] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:10.715448 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c34a5-f216-48da-9e3c-3afeaad6d6e9 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:10.726750 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c34a5-f216-48da-9e3c-3afeaad6d6e9 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:10.732964 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a23c68-2121-4d26-a47f-24a94f5b1c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:10.733710 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a23c68-2121-4d26-a47f-24a94f5b1c3b admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:10.734030 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a23c68-2121-4d26-a47f-24a94f5b1c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:10.734414 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a23c68-2121-4d26-a47f-24a94f5b1c3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:10.735727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a23c68-2121-4d26-a47f-24a94f5b1c3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:10.737900 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a23c68-2121-4d26-a47f-24a94f5b1c3b admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:10.808252 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 973/1924] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:13:10 2022] POST /identity/v3/projects => generated 373 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:10.813514 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea506f3-1ab7-420f-b4a3-8be9eca1f18e service neutron] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:10.846093 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea506f3-1ab7-420f-b4a3-8be9eca1f18e service neutron] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:11.071980 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a23c68-2121-4d26-a47f-24a94f5b1c3b admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:11.078302 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-d8a23c68-2121-4d26-a47f-24a94f5b1c3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:11.213499 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f278a-b09d-4ee5-a88d-f06fac6892b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:11.217857 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f278a-b09d-4ee5-a88d-f06fac6892b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:11.218442 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f278a-b09d-4ee5-a88d-f06fac6892b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:11.220184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f278a-b09d-4ee5-a88d-f06fac6892b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:11.224379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f278a-b09d-4ee5-a88d-f06fac6892b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:11.235930 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f278a-b09d-4ee5-a88d-f06fac6892b0 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:11.241113 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:11.273517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:11.321082 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 952/1925] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:13:10 2022] POST /identity/v3/users => generated 287 bytes in 509 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:11.322130 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f278a-b09d-4ee5-a88d-f06fac6892b0 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:11.350153 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f278a-b09d-4ee5-a88d-f06fac6892b0 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:11.367225 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14ee99-eab6-43dc-9776-da56202598db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:11.367910 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14ee99-eab6-43dc-9776-da56202598db admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:11.368192 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14ee99-eab6-43dc-9776-da56202598db admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:11.368462 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14ee99-eab6-43dc-9776-da56202598db admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:11.369162 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14ee99-eab6-43dc-9776-da56202598db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:11.370921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14ee99-eab6-43dc-9776-da56202598db admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:11.376262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a23c68-2121-4d26-a47f-24a94f5b1c3b admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:11.388669 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 953/1926] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:11.391551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfff98-c529-4f71-b5b1-cb317b9920a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:11.392686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14ee99-eab6-43dc-9776-da56202598db admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:11.394067 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfff98-c529-4f71-b5b1-cb317b9920a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:11.394904 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfff98-c529-4f71-b5b1-cb317b9920a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:11.396918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfff98-c529-4f71-b5b1-cb317b9920a4 admin admin] PATH_INFO: `/v3/users/8161565be48d4c32b0efc3d287b286a0` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:11.403065 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfff98-c529-4f71-b5b1-cb317b9920a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8161565be48d4c32b0efc3d287b286a0)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:11.405285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfff98-c529-4f71-b5b1-cb317b9920a4 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:11.414423 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14ee99-eab6-43dc-9776-da56202598db admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:11.433850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cea7e6-e15a-4842-925d-4b2f89d60089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:11.434483 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cea7e6-e15a-4842-925d-4b2f89d60089 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:11.434760 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cea7e6-e15a-4842-925d-4b2f89d60089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:11.435030 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cea7e6-e15a-4842-925d-4b2f89d60089 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:11.435691 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cea7e6-e15a-4842-925d-4b2f89d60089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:11.437381 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cea7e6-e15a-4842-925d-4b2f89d60089 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:11.470504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 954/1927] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:11.474538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cea7e6-e15a-4842-925d-4b2f89d60089 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:11.487780 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cea7e6-e15a-4842-925d-4b2f89d60089 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:11.501969 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-46bfff98-c529-4f71-b5b1-cb317b9920a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8161565be48d4c32b0efc3d287b286a0'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:11.505998 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d8dde-0757-40fd-b957-b28e9cedf603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:11.507328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d8dde-0757-40fd-b957-b28e9cedf603 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:11.507328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d8dde-0757-40fd-b957-b28e9cedf603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:11.507502 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d8dde-0757-40fd-b957-b28e9cedf603 admin admin] PATH_INFO: `/v3/projects/3b37cd218e1f4a8181be9f8eac809535/users/0f2c82c962454acc90fe051746b6fceb/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:11.520616 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-46bfff98-c529-4f71-b5b1-cb317b9920a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8161565be48d4c32b0efc3d287b286a0'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:11.540195 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-46bfff98-c529-4f71-b5b1-cb317b9920a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8161565be48d4c32b0efc3d287b286a0'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:11.541075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-46bfff98-c529-4f71-b5b1-cb317b9920a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8161565be48d4c32b0efc3d287b286a0'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:11.547613 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d8dde-0757-40fd-b957-b28e9cedf603 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b37cd218e1f4a8181be9f8eac809535, user_id=0f2c82c962454acc90fe051746b6fceb, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:11.555290 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-46bfff98-c529-4f71-b5b1-cb317b9920a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8161565be48d4c32b0efc3d287b286a0'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:11.555543 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d8dde-0757-40fd-b957-b28e9cedf603 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:11.585815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 974/1928] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:11 2022] DELETE /identity/v3/users/8161565be48d4c32b0efc3d287b286a0 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:11.588480 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f752d6-3408-41bd-a62c-d52e399f108a None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:11.588766 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f752d6-3408-41bd-a62c-d52e399f108a None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:11.589027 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f752d6-3408-41bd-a62c-d52e399f108a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:11.598321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 955/1929] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:11 2022] PUT /identity/v3/projects/3b37cd218e1f4a8181be9f8eac809535/users/0f2c82c962454acc90fe051746b6fceb/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:13:11.600272 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86828a4e-18e0-48ed-9be5-210a9cfb23b1 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:11.600562 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86828a4e-18e0-48ed-9be5-210a9cfb23b1 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:11.600832 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86828a4e-18e0-48ed-9be5-210a9cfb23b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:11.648886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-46f752d6-3408-41bd-a62c-d52e399f108a None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:11.649992 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-46f752d6-3408-41bd-a62c-d52e399f108a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:11.658017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-86828a4e-18e0-48ed-9be5-210a9cfb23b1 None None] MFA Rules not processed for user `0f2c82c962454acc90fe051746b6fceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:11.665098 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-86828a4e-18e0-48ed-9be5-210a9cfb23b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:11.828532 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 956/1930] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:13:11.831354 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86828a4e-18e0-48ed-9be5-210a9cfb23b1 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:11.833927 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-86828a4e-18e0-48ed-9be5-210a9cfb23b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:11.851048 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 975/1931] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:11.851805 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f752d6-3408-41bd-a62c-d52e399f108a None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:11.855766 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-46f752d6-3408-41bd-a62c-d52e399f108a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:11.864900 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86828a4e-18e0-48ed-9be5-210a9cfb23b1 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:11.874274 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd499b-919b-4ed3-b8ed-c472b2484594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:11.875357 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd499b-919b-4ed3-b8ed-c472b2484594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:11.878793 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd499b-919b-4ed3-b8ed-c472b2484594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:11.878793 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd499b-919b-4ed3-b8ed-c472b2484594 admin admin] PATH_INFO: `/v3/projects/5b1e9b6b29bd4dc49b05acd1764e0a40` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:11.884918 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd499b-919b-4ed3-b8ed-c472b2484594 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b1e9b6b29bd4dc49b05acd1764e0a40)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:11.887256 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd499b-919b-4ed3-b8ed-c472b2484594 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:11.906354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f752d6-3408-41bd-a62c-d52e399f108a None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:11.916579 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ceacb-a132-4a49-bef3-36c1632e7b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:11.917249 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ceacb-a132-4a49-bef3-36c1632e7b59 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:11.917537 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ceacb-a132-4a49-bef3-36c1632e7b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:11.917969 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ceacb-a132-4a49-bef3-36c1632e7b59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:11.918778 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ceacb-a132-4a49-bef3-36c1632e7b59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:11.921778 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ceacb-a132-4a49-bef3-36c1632e7b59 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:11.973090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 976/1932] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:13:11 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:11.974150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ceacb-a132-4a49-bef3-36c1632e7b59 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:11.976711 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-754ceacb-a132-4a49-bef3-36c1632e7b59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:12.013934 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-a1bd499b-919b-4ed3-b8ed-c472b2484594 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b1e9b6b29bd4dc49b05acd1764e0a40', 'request_id': 'req-a1bd499b-919b-4ed3-b8ed-c472b2484594'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:12.028353 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:12.034623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-a1bd499b-919b-4ed3-b8ed-c472b2484594 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b1e9b6b29bd4dc49b05acd1764e0a40', 'request_id': 'req-a1bd499b-919b-4ed3-b8ed-c472b2484594'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:12.047461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-a1bd499b-919b-4ed3-b8ed-c472b2484594 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b1e9b6b29bd4dc49b05acd1764e0a40', 'request_id': 'req-a1bd499b-919b-4ed3-b8ed-c472b2484594'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:12.077589 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-a1bd499b-919b-4ed3-b8ed-c472b2484594 admin admin] The token cache is being invalidate because project 5b1e9b6b29bd4dc49b05acd1764e0a40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:12.078653 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-a1bd499b-919b-4ed3-b8ed-c472b2484594 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:12.079774 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-a1bd499b-919b-4ed3-b8ed-c472b2484594 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:12.086283 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 957/1933] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:11 2022] DELETE /identity/v3/projects/5b1e9b6b29bd4dc49b05acd1764e0a40 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:12.088544 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd499b-919b-4ed3-b8ed-c472b2484594 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:12.091929 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-a1bd499b-919b-4ed3-b8ed-c472b2484594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:12.104124 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ceacb-a132-4a49-bef3-36c1632e7b59 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:12.110426 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca41978-f84e-44bc-9ec8-2a922e31aff7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:12.113905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca41978-f84e-44bc-9ec8-2a922e31aff7 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:12.114184 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca41978-f84e-44bc-9ec8-2a922e31aff7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:12.114451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca41978-f84e-44bc-9ec8-2a922e31aff7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:12.122000 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-aca41978-f84e-44bc-9ec8-2a922e31aff7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:12.145484 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:12.262920 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd499b-919b-4ed3-b8ed-c472b2484594 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:12.267087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca41978-f84e-44bc-9ec8-2a922e31aff7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:12.268716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca41978-f84e-44bc-9ec8-2a922e31aff7 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:12.289889 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 977/1934] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:12.291022 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca41978-f84e-44bc-9ec8-2a922e31aff7 service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:12.292699 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa872f-8484-4e08-bf5f-208b5c79fdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:12.293370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa872f-8484-4e08-bf5f-208b5c79fdcd admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:12.293479 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-aca41978-f84e-44bc-9ec8-2a922e31aff7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:12.293664 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa872f-8484-4e08-bf5f-208b5c79fdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:12.293912 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa872f-8484-4e08-bf5f-208b5c79fdcd admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:12.294554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa872f-8484-4e08-bf5f-208b5c79fdcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:12.301453 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa872f-8484-4e08-bf5f-208b5c79fdcd admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:12.307902 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca41978-f84e-44bc-9ec8-2a922e31aff7 service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:12.314026 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8e7fb-ac43-46d7-83b8-ff715d13b34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:12.314712 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8e7fb-ac43-46d7-83b8-ff715d13b34c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:12.315064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8e7fb-ac43-46d7-83b8-ff715d13b34c admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:12.315250 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8e7fb-ac43-46d7-83b8-ff715d13b34c admin admin] PATH_INFO: `/v3/users/b872ce80cf3147ea8f31d63a137b3250` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:12.316611 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8e7fb-ac43-46d7-83b8-ff715d13b34c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b872ce80cf3147ea8f31d63a137b3250)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:12.319522 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8e7fb-ac43-46d7-83b8-ff715d13b34c admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:12.392633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-50d8e7fb-ac43-46d7-83b8-ff715d13b34c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b872ce80cf3147ea8f31d63a137b3250'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:12.409629 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-50d8e7fb-ac43-46d7-83b8-ff715d13b34c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b872ce80cf3147ea8f31d63a137b3250'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:12.415492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 958/1935] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:13:12 2022] POST /identity/v3/users => generated 294 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:12.417200 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa872f-8484-4e08-bf5f-208b5c79fdcd admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:12.431507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa872f-8484-4e08-bf5f-208b5c79fdcd admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:12.431703 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-50d8e7fb-ac43-46d7-83b8-ff715d13b34c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b872ce80cf3147ea8f31d63a137b3250'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:12.432366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-50d8e7fb-ac43-46d7-83b8-ff715d13b34c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b872ce80cf3147ea8f31d63a137b3250'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:12.438624 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f175d-94e4-47e7-b9bb-7e97a5c58461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:12.439812 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f175d-94e4-47e7-b9bb-7e97a5c58461 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:12.440259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f175d-94e4-47e7-b9bb-7e97a5c58461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:12.440531 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f175d-94e4-47e7-b9bb-7e97a5c58461 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:12.441218 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f175d-94e4-47e7-b9bb-7e97a5c58461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:12.441444 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-50d8e7fb-ac43-46d7-83b8-ff715d13b34c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b872ce80cf3147ea8f31d63a137b3250'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:12.443673 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f175d-94e4-47e7-b9bb-7e97a5c58461 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:12.454493 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 978/1936] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:12 2022] DELETE /identity/v3/users/b872ce80cf3147ea8f31d63a137b3250 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:12.463942 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 959/1937] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:12.468596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8e7fb-ac43-46d7-83b8-ff715d13b34c admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:12.470439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-50d8e7fb-ac43-46d7-83b8-ff715d13b34c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:12.544135 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:12.586522 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8e7fb-ac43-46d7-83b8-ff715d13b34c admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:12.595083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7eeacd-cab6-4f20-978d-d4351fe89f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:12.596912 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7eeacd-cab6-4f20-978d-d4351fe89f32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:12.597935 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7eeacd-cab6-4f20-978d-d4351fe89f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:12.597935 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7eeacd-cab6-4f20-978d-d4351fe89f32 admin admin] PATH_INFO: `/v3/projects/c56fc966b38646c6aaf97b353ca76190/users/1e3628e18ff240ca80707e1ac238a6a2/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:12.623818 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7eeacd-cab6-4f20-978d-d4351fe89f32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c56fc966b38646c6aaf97b353ca76190, user_id=1e3628e18ff240ca80707e1ac238a6a2, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:12.627416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7eeacd-cab6-4f20-978d-d4351fe89f32 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:12.655173 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 979/1938] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:12 2022] PUT /identity/v3/projects/c56fc966b38646c6aaf97b353ca76190/users/1e3628e18ff240ca80707e1ac238a6a2/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:12.665263 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f175d-94e4-47e7-b9bb-7e97a5c58461 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:12.684330 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f175d-94e4-47e7-b9bb-7e97a5c58461 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:12.795292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15758d2c-7dfc-437d-b29f-42bc6a003b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:12.796174 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15758d2c-7dfc-437d-b29f-42bc6a003b91 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:12.797398 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15758d2c-7dfc-437d-b29f-42bc6a003b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:12.797740 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15758d2c-7dfc-437d-b29f-42bc6a003b91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:12.798681 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15758d2c-7dfc-437d-b29f-42bc6a003b91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:12.802981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15758d2c-7dfc-437d-b29f-42bc6a003b91 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:12.821623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 960/1939] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:12.823735 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7eeacd-cab6-4f20-978d-d4351fe89f32 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:12.835993 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15758d2c-7dfc-437d-b29f-42bc6a003b91 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:12.838458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-15758d2c-7dfc-437d-b29f-42bc6a003b91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:12.840556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7eeacd-cab6-4f20-978d-d4351fe89f32 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:12.847029 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2673c5-42d4-411c-a5d8-c3f3748988b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:12.856345 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15758d2c-7dfc-437d-b29f-42bc6a003b91 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:12.857655 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2673c5-42d4-411c-a5d8-c3f3748988b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:12.858151 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2673c5-42d4-411c-a5d8-c3f3748988b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:12.858623 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2673c5-42d4-411c-a5d8-c3f3748988b6 admin admin] PATH_INFO: `/v3/projects/c56fc966b38646c6aaf97b353ca76190/users/1e3628e18ff240ca80707e1ac238a6a2/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:12.860888 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2673c5-42d4-411c-a5d8-c3f3748988b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c56fc966b38646c6aaf97b353ca76190, user_id=1e3628e18ff240ca80707e1ac238a6a2, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:12.861738 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a725b-3fea-4c7e-ae82-c5d65d2a5eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:12.862496 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a725b-3fea-4c7e-ae82-c5d65d2a5eca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:12.863090 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a725b-3fea-4c7e-ae82-c5d65d2a5eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:12.863378 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a725b-3fea-4c7e-ae82-c5d65d2a5eca admin admin] PATH_INFO: `/v3/projects/564ab122f1c0461987ab59f6d259b769` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:12.864002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2673c5-42d4-411c-a5d8-c3f3748988b6 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:12.864804 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a725b-3fea-4c7e-ae82-c5d65d2a5eca admin admin] RBAC: Authorizing `identity:delete_project(project_id=564ab122f1c0461987ab59f6d259b769)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:12.866872 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a725b-3fea-4c7e-ae82-c5d65d2a5eca admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:12.885884 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 980/1940] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:12 2022] PUT /identity/v3/projects/c56fc966b38646c6aaf97b353ca76190/users/1e3628e18ff240ca80707e1ac238a6a2/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:12.886986 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2673c5-42d4-411c-a5d8-c3f3748988b6 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:12.889499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-df2673c5-42d4-411c-a5d8-c3f3748988b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:12.941005 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-b82a725b-3fea-4c7e-ae82-c5d65d2a5eca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '564ab122f1c0461987ab59f6d259b769', 'request_id': 'req-b82a725b-3fea-4c7e-ae82-c5d65d2a5eca'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:12.949115 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-b82a725b-3fea-4c7e-ae82-c5d65d2a5eca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '564ab122f1c0461987ab59f6d259b769', 'request_id': 'req-b82a725b-3fea-4c7e-ae82-c5d65d2a5eca'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:12.964308 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-b82a725b-3fea-4c7e-ae82-c5d65d2a5eca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '564ab122f1c0461987ab59f6d259b769', 'request_id': 'req-b82a725b-3fea-4c7e-ae82-c5d65d2a5eca'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:12.975644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2673c5-42d4-411c-a5d8-c3f3748988b6 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:12.979622 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-b82a725b-3fea-4c7e-ae82-c5d65d2a5eca admin admin] The token cache is being invalidate because project 564ab122f1c0461987ab59f6d259b769 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:12.979921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-b82a725b-3fea-4c7e-ae82-c5d65d2a5eca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:12.980606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-b82a725b-3fea-4c7e-ae82-c5d65d2a5eca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:12.983900 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 961/1941] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:12 2022] DELETE /identity/v3/projects/564ab122f1c0461987ab59f6d259b769 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:12.984373 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c6099-6180-4783-8139-6124e8e111cf None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:12.984654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c6099-6180-4783-8139-6124e8e111cf None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:12.984938 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c6099-6180-4783-8139-6124e8e111cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:13.020957 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:13.026443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-003c6099-6180-4783-8139-6124e8e111cf None None] MFA Rules not processed for user `1e3628e18ff240ca80707e1ac238a6a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:13.028062 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-003c6099-6180-4783-8139-6124e8e111cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:13.062985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc09a5d-6401-49ac-bca3-5559b5872b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:13.064794 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc09a5d-6401-49ac-bca3-5559b5872b1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:13.065187 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc09a5d-6401-49ac-bca3-5559b5872b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:13.065811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc09a5d-6401-49ac-bca3-5559b5872b1a admin admin] PATH_INFO: `/v3/projects/7fa5862a888d42fc8f85249f36c1a403` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:13.067769 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc09a5d-6401-49ac-bca3-5559b5872b1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fa5862a888d42fc8f85249f36c1a403)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:13.070140 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc09a5d-6401-49ac-bca3-5559b5872b1a admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:13.150559 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 962/1942] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:13.151227 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c6099-6180-4783-8139-6124e8e111cf None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:13.154877 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-003c6099-6180-4783-8139-6124e8e111cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:13.155578 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2dc09a5d-6401-49ac-bca3-5559b5872b1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fa5862a888d42fc8f85249f36c1a403', 'request_id': 'req-2dc09a5d-6401-49ac-bca3-5559b5872b1a'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:13.165157 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2dc09a5d-6401-49ac-bca3-5559b5872b1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fa5862a888d42fc8f85249f36c1a403', 'request_id': 'req-2dc09a5d-6401-49ac-bca3-5559b5872b1a'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:13.176001 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2dc09a5d-6401-49ac-bca3-5559b5872b1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fa5862a888d42fc8f85249f36c1a403', 'request_id': 'req-2dc09a5d-6401-49ac-bca3-5559b5872b1a'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:13.180256 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:13.186090 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c6099-6180-4783-8139-6124e8e111cf None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:13.204913 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2dc09a5d-6401-49ac-bca3-5559b5872b1a admin admin] The token cache is being invalidate because project 7fa5862a888d42fc8f85249f36c1a403 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:13.205844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2dc09a5d-6401-49ac-bca3-5559b5872b1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:13.207094 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2dc09a5d-6401-49ac-bca3-5559b5872b1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:13.209535 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 981/1943] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:12 2022] DELETE /identity/v3/projects/7fa5862a888d42fc8f85249f36c1a403 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:13.216648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc09a5d-6401-49ac-bca3-5559b5872b1a admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:13.218093 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-2dc09a5d-6401-49ac-bca3-5559b5872b1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:13.302156 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cef1ab-96e3-415c-aa43-f209bf39448f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:13.302811 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cef1ab-96e3-415c-aa43-f209bf39448f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:13.303158 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cef1ab-96e3-415c-aa43-f209bf39448f admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:13.303416 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cef1ab-96e3-415c-aa43-f209bf39448f admin admin] PATH_INFO: `/v3/users/83e6a62d72d54fbdac8815d1cb94d9a0` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:13.304720 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cef1ab-96e3-415c-aa43-f209bf39448f admin admin] RBAC: Authorizing `identity:delete_user(user_id=83e6a62d72d54fbdac8815d1cb94d9a0)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:13.306726 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cef1ab-96e3-415c-aa43-f209bf39448f admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:13.317572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc09a5d-6401-49ac-bca3-5559b5872b1a admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:13.322994 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291722a2-8f08-440c-b492-2c855e65bafe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:13.324701 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291722a2-8f08-440c-b492-2c855e65bafe service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:13.325488 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291722a2-8f08-440c-b492-2c855e65bafe service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:13.325904 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291722a2-8f08-440c-b492-2c855e65bafe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:13.332280 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-291722a2-8f08-440c-b492-2c855e65bafe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:13.371555 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-49cef1ab-96e3-415c-aa43-f209bf39448f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83e6a62d72d54fbdac8815d1cb94d9a0'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:13.387936 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-49cef1ab-96e3-415c-aa43-f209bf39448f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83e6a62d72d54fbdac8815d1cb94d9a0'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:13.414665 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-49cef1ab-96e3-415c-aa43-f209bf39448f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83e6a62d72d54fbdac8815d1cb94d9a0'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:13.418890 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-49cef1ab-96e3-415c-aa43-f209bf39448f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83e6a62d72d54fbdac8815d1cb94d9a0'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:13.434298 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291722a2-8f08-440c-b492-2c855e65bafe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:13.435764 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291722a2-8f08-440c-b492-2c855e65bafe service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:13.449804 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 982/1944] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:13.452343 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec151069-b85a-4e14-a5ab-e67cdabd76f9 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:13.453193 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec151069-b85a-4e14-a5ab-e67cdabd76f9 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:13.454105 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-49cef1ab-96e3-415c-aa43-f209bf39448f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83e6a62d72d54fbdac8815d1cb94d9a0'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:13.455383 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec151069-b85a-4e14-a5ab-e67cdabd76f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:13.474720 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 963/1945] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:13 2022] DELETE /identity/v3/users/83e6a62d72d54fbdac8815d1cb94d9a0 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:13.507411 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-ec151069-b85a-4e14-a5ab-e67cdabd76f9 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:13.508881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-ec151069-b85a-4e14-a5ab-e67cdabd76f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:13.597461 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 983/1946] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:13.605867 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170d436-081e-45dd-9b1c-25c11035adf2 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:13.606151 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170d436-081e-45dd-9b1c-25c11035adf2 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:13.606427 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170d436-081e-45dd-9b1c-25c11035adf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:13.635315 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-6170d436-081e-45dd-9b1c-25c11035adf2 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:13.636540 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-6170d436-081e-45dd-9b1c-25c11035adf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:13.646460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 964/1947] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:13.653242 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec151069-b85a-4e14-a5ab-e67cdabd76f9 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:13.669029 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec151069-b85a-4e14-a5ab-e67cdabd76f9 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:13.683992 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85227c46-4e7b-4c16-86ed-952e19567604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:13.684792 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85227c46-4e7b-4c16-86ed-952e19567604 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:13.685189 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85227c46-4e7b-4c16-86ed-952e19567604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:13.685582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85227c46-4e7b-4c16-86ed-952e19567604 admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:13.686405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85227c46-4e7b-4c16-86ed-952e19567604 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:13:13.686782 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85227c46-4e7b-4c16-86ed-952e19567604 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:13.688487 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85227c46-4e7b-4c16-86ed-952e19567604 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:13.710020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170d436-081e-45dd-9b1c-25c11035adf2 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:13.717556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 984/1948] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:13:13.722144 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170d436-081e-45dd-9b1c-25c11035adf2 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:13.736860 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ed34c-f610-4d0a-ae59-a95885ff1544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:13.737180 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ed34c-f610-4d0a-ae59-a95885ff1544 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:13.737575 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ed34c-f610-4d0a-ae59-a95885ff1544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:13.738121 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ed34c-f610-4d0a-ae59-a95885ff1544 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:13.739353 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ed34c-f610-4d0a-ae59-a95885ff1544 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:13.741745 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ed34c-f610-4d0a-ae59-a95885ff1544 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:13.758275 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:13.817239 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 965/1949] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:13:13 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:13.823339 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85227c46-4e7b-4c16-86ed-952e19567604 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:13.869702 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85227c46-4e7b-4c16-86ed-952e19567604 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:13.935857 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:14.033791 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a119dac-9f5d-4bc3-960b-e4a83129c007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:14.034843 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a119dac-9f5d-4bc3-960b-e4a83129c007 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:14.035338 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a119dac-9f5d-4bc3-960b-e4a83129c007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:14.035814 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a119dac-9f5d-4bc3-960b-e4a83129c007 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:14.036936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a119dac-9f5d-4bc3-960b-e4a83129c007 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:14.043579 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a119dac-9f5d-4bc3-960b-e4a83129c007 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:14.097708 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 985/1950] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:13:13 2022] POST /identity/v3/users => generated 283 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:13:14.103532 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ed34c-f610-4d0a-ae59-a95885ff1544 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:14.118475 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ed34c-f610-4d0a-ae59-a95885ff1544 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:14.131139 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a5d31b-98a4-4fcc-85e7-66f355292193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:14.133244 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5d31b-98a4-4fcc-85e7-66f355292193 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:14.133610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5d31b-98a4-4fcc-85e7-66f355292193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:14.133951 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5d31b-98a4-4fcc-85e7-66f355292193 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:14.134699 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a5d31b-98a4-4fcc-85e7-66f355292193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:14.136600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a5d31b-98a4-4fcc-85e7-66f355292193 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:14.154037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 966/1951] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:14.158213 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a119dac-9f5d-4bc3-960b-e4a83129c007 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:14.170270 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a119dac-9f5d-4bc3-960b-e4a83129c007 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:14.175149 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79dc75-7e04-421e-b33f-13491f2a03db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:14.175617 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79dc75-7e04-421e-b33f-13491f2a03db admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:14.175936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79dc75-7e04-421e-b33f-13491f2a03db admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:14.176519 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79dc75-7e04-421e-b33f-13491f2a03db admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:14.177356 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79dc75-7e04-421e-b33f-13491f2a03db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:14.178917 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79dc75-7e04-421e-b33f-13491f2a03db admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:14.193261 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 986/1952] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:14.196346 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a5d31b-98a4-4fcc-85e7-66f355292193 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:14.208207 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a5d31b-98a4-4fcc-85e7-66f355292193 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:14.213666 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676c51f-3284-4ebe-9109-88940ecbaf13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:14.214348 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676c51f-3284-4ebe-9109-88940ecbaf13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:14.214741 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676c51f-3284-4ebe-9109-88940ecbaf13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:14.214912 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676c51f-3284-4ebe-9109-88940ecbaf13 admin admin] PATH_INFO: `/v3/projects/b401a6807f1645339ef62198d9a55ab7/users/a380060812584f83b0655eaa67badab4/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:14.219822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79dc75-7e04-421e-b33f-13491f2a03db admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:14.221732 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-8e79dc75-7e04-421e-b33f-13491f2a03db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:14.236714 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79dc75-7e04-421e-b33f-13491f2a03db admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:14.239883 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676c51f-3284-4ebe-9109-88940ecbaf13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b401a6807f1645339ef62198d9a55ab7, user_id=a380060812584f83b0655eaa67badab4, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:14.243601 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ad294-ce1d-4cdb-a0c5-4d99f5f46b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:14.244028 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676c51f-3284-4ebe-9109-88940ecbaf13 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:14.244246 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ad294-ce1d-4cdb-a0c5-4d99f5f46b70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:14.244628 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ad294-ce1d-4cdb-a0c5-4d99f5f46b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:14.245022 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ad294-ce1d-4cdb-a0c5-4d99f5f46b70 admin admin] PATH_INFO: `/v3/projects/f361435efba64cf1a832bd250dcc59ef` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:14.246264 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ad294-ce1d-4cdb-a0c5-4d99f5f46b70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f361435efba64cf1a832bd250dcc59ef)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:14.254641 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ad294-ce1d-4cdb-a0c5-4d99f5f46b70 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:14.269174 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 967/1953] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:14 2022] PUT /identity/v3/projects/b401a6807f1645339ef62198d9a55ab7/users/a380060812584f83b0655eaa67badab4/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:14.280190 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295511d-566d-4c4d-87e5-c57eb1f7f68f None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:14.280468 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295511d-566d-4c4d-87e5-c57eb1f7f68f None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:14.282101 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295511d-566d-4c4d-87e5-c57eb1f7f68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:14.317456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-6295511d-566d-4c4d-87e5-c57eb1f7f68f None None] MFA Rules not processed for user `a380060812584f83b0655eaa67badab4`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:14.318761 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-6295511d-566d-4c4d-87e5-c57eb1f7f68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:14.328181 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-b22ad294-ce1d-4cdb-a0c5-4d99f5f46b70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f361435efba64cf1a832bd250dcc59ef', 'request_id': 'req-b22ad294-ce1d-4cdb-a0c5-4d99f5f46b70'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:14.336217 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-b22ad294-ce1d-4cdb-a0c5-4d99f5f46b70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f361435efba64cf1a832bd250dcc59ef', 'request_id': 'req-b22ad294-ce1d-4cdb-a0c5-4d99f5f46b70'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:14.345968 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-b22ad294-ce1d-4cdb-a0c5-4d99f5f46b70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f361435efba64cf1a832bd250dcc59ef', 'request_id': 'req-b22ad294-ce1d-4cdb-a0c5-4d99f5f46b70'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:14.370019 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-b22ad294-ce1d-4cdb-a0c5-4d99f5f46b70 admin admin] The token cache is being invalidate because project f361435efba64cf1a832bd250dcc59ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:14.370355 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-b22ad294-ce1d-4cdb-a0c5-4d99f5f46b70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:14.371012 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-b22ad294-ce1d-4cdb-a0c5-4d99f5f46b70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:14.373659 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 987/1954] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:14 2022] DELETE /identity/v3/projects/f361435efba64cf1a832bd250dcc59ef => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:13:14.377834 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ad294-ce1d-4cdb-a0c5-4d99f5f46b70 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:14.379741 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-b22ad294-ce1d-4cdb-a0c5-4d99f5f46b70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:14.402837 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 968/1955] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:14.407415 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6295511d-566d-4c4d-87e5-c57eb1f7f68f None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:14.413330 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-6295511d-566d-4c4d-87e5-c57eb1f7f68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:14.471474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ad294-ce1d-4cdb-a0c5-4d99f5f46b70 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:14.478016 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b5d623-cd1d-47d3-8eb8-eadfb8b6d4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:14.478698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5d623-cd1d-47d3-8eb8-eadfb8b6d4f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:14.483657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5d623-cd1d-47d3-8eb8-eadfb8b6d4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:14.483657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5d623-cd1d-47d3-8eb8-eadfb8b6d4f8 admin admin] PATH_INFO: `/v3/users/00fa577ef860438e8795b9ab3424205b` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:14.483657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b5d623-cd1d-47d3-8eb8-eadfb8b6d4f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00fa577ef860438e8795b9ab3424205b)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:14.485397 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b5d623-cd1d-47d3-8eb8-eadfb8b6d4f8 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:14.546536 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=84,keystone:INSERT=4,keystone:DELETE=12 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:14.546743 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6295511d-566d-4c4d-87e5-c57eb1f7f68f None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:14.551477 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-a3b5d623-cd1d-47d3-8eb8-eadfb8b6d4f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00fa577ef860438e8795b9ab3424205b'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:14.572477 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c19449-5468-48dc-a7d4-590e4c0b074e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:14.573554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c19449-5468-48dc-a7d4-590e4c0b074e admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:14.574652 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c19449-5468-48dc-a7d4-590e4c0b074e admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:14.575675 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c19449-5468-48dc-a7d4-590e4c0b074e admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:14.576793 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c19449-5468-48dc-a7d4-590e4c0b074e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:14.580477 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-a3b5d623-cd1d-47d3-8eb8-eadfb8b6d4f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00fa577ef860438e8795b9ab3424205b'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:14.580815 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c19449-5468-48dc-a7d4-590e4c0b074e admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:14.604449 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-a3b5d623-cd1d-47d3-8eb8-eadfb8b6d4f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00fa577ef860438e8795b9ab3424205b'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:14.614128 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-a3b5d623-cd1d-47d3-8eb8-eadfb8b6d4f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00fa577ef860438e8795b9ab3424205b'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:14.631069 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 969/1956] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:13:14 2022] POST /identity/v3/projects => generated 363 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:14.638146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-a3b5d623-cd1d-47d3-8eb8-eadfb8b6d4f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00fa577ef860438e8795b9ab3424205b'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:14.638600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c19449-5468-48dc-a7d4-590e4c0b074e admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:14.640978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-02c19449-5468-48dc-a7d4-590e4c0b074e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:14.651870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 988/1957] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:14 2022] DELETE /identity/v3/users/00fa577ef860438e8795b9ab3424205b => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:14.684147 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:14.728020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c19449-5468-48dc-a7d4-590e4c0b074e admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:14.800208 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015cdab6-e830-469e-88b4-2dd6a1eb7d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:14.800930 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cdab6-e830-469e-88b4-2dd6a1eb7d8c admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:14.801272 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cdab6-e830-469e-88b4-2dd6a1eb7d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:14.801661 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cdab6-e830-469e-88b4-2dd6a1eb7d8c admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:14.802307 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015cdab6-e830-469e-88b4-2dd6a1eb7d8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:14.804366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015cdab6-e830-469e-88b4-2dd6a1eb7d8c admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:14.828105 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 970/1958] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:13:14 2022] POST /identity/v3/users => generated 288 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:13:14.832385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b5d623-cd1d-47d3-8eb8-eadfb8b6d4f8 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:14.843393 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b5d623-cd1d-47d3-8eb8-eadfb8b6d4f8 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:14.849221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971e7dc-3c96-4123-84d1-7939b8eb2fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:14.849844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971e7dc-3c96-4123-84d1-7939b8eb2fbb admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:14.850208 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971e7dc-3c96-4123-84d1-7939b8eb2fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:14.850464 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971e7dc-3c96-4123-84d1-7939b8eb2fbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:14.851116 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8971e7dc-3c96-4123-84d1-7939b8eb2fbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:14.852748 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8971e7dc-3c96-4123-84d1-7939b8eb2fbb admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:14.863948 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 989/1959] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:13:14.868073 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015cdab6-e830-469e-88b4-2dd6a1eb7d8c admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:14.878606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015cdab6-e830-469e-88b4-2dd6a1eb7d8c admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:14.884544 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea25a4e-0601-4904-8df7-b33a24d0a41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:14.885300 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea25a4e-0601-4904-8df7-b33a24d0a41e admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:14.885578 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea25a4e-0601-4904-8df7-b33a24d0a41e admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:14.886166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea25a4e-0601-4904-8df7-b33a24d0a41e admin admin] PATH_INFO: `/v3/projects/e651265c6b4a4383aec48d23fc0b7776/users/1255b48fe9c0435e8de043262fe72370/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:14.925670 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea25a4e-0601-4904-8df7-b33a24d0a41e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e651265c6b4a4383aec48d23fc0b7776, user_id=1255b48fe9c0435e8de043262fe72370, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:14.930197 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea25a4e-0601-4904-8df7-b33a24d0a41e admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:14.958784 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 971/1960] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:14 2022] PUT /identity/v3/projects/e651265c6b4a4383aec48d23fc0b7776/users/1255b48fe9c0435e8de043262fe72370/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:14.962711 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971e7dc-3c96-4123-84d1-7939b8eb2fbb admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:14.980839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971e7dc-3c96-4123-84d1-7939b8eb2fbb admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:15.101369 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d49b9-4092-4bec-a9c9-fc45d3279246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:15.102097 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d49b9-4092-4bec-a9c9-fc45d3279246 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:15.102434 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d49b9-4092-4bec-a9c9-fc45d3279246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:15.102762 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d49b9-4092-4bec-a9c9-fc45d3279246 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:15.103939 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d49b9-4092-4bec-a9c9-fc45d3279246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:15.105385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d49b9-4092-4bec-a9c9-fc45d3279246 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:15.137440 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea25a4e-0601-4904-8df7-b33a24d0a41e admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:15.143097 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 990/1961] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:15.145698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-fea25a4e-0601-4904-8df7-b33a24d0a41e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:15.145898 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d49b9-4092-4bec-a9c9-fc45d3279246 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:15.171110 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d49b9-4092-4bec-a9c9-fc45d3279246 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:15.177524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf637e3-4746-4ade-9b4d-b1b29e962112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:15.178270 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf637e3-4746-4ade-9b4d-b1b29e962112 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:15.178638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf637e3-4746-4ade-9b4d-b1b29e962112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:15.179297 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf637e3-4746-4ade-9b4d-b1b29e962112 admin admin] PATH_INFO: `/v3/projects/e651265c6b4a4383aec48d23fc0b7776/users/1255b48fe9c0435e8de043262fe72370/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:15.184737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea25a4e-0601-4904-8df7-b33a24d0a41e admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:15.196830 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf637e3-4746-4ade-9b4d-b1b29e962112 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e651265c6b4a4383aec48d23fc0b7776, user_id=1255b48fe9c0435e8de043262fe72370, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:15.208095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf637e3-4746-4ade-9b4d-b1b29e962112 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:15.208988 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d99b1ed-bd27-4a4a-96d2-242ba8f97b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:15.210214 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99b1ed-bd27-4a4a-96d2-242ba8f97b31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:15.210804 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99b1ed-bd27-4a4a-96d2-242ba8f97b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:15.211369 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99b1ed-bd27-4a4a-96d2-242ba8f97b31 admin admin] PATH_INFO: `/v3/projects/ef579a625f6a46b39b8c21fabed3eb7c` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:15.213028 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d99b1ed-bd27-4a4a-96d2-242ba8f97b31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef579a625f6a46b39b8c21fabed3eb7c)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:15.216631 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d99b1ed-bd27-4a4a-96d2-242ba8f97b31 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:15.268211 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 991/1962] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:15 2022] PUT /identity/v3/projects/e651265c6b4a4383aec48d23fc0b7776/users/1255b48fe9c0435e8de043262fe72370/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:13:15.279702 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b2964-7db3-4631-8738-4ca4fc445fad None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:15.280020 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b2964-7db3-4631-8738-4ca4fc445fad None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:15.280576 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b2964-7db3-4631-8738-4ca4fc445fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:15.331439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-169b2964-7db3-4631-8738-4ca4fc445fad None None] MFA Rules not processed for user `1255b48fe9c0435e8de043262fe72370`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:15.332706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-169b2964-7db3-4631-8738-4ca4fc445fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:15.333517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-9d99b1ed-bd27-4a4a-96d2-242ba8f97b31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef579a625f6a46b39b8c21fabed3eb7c', 'request_id': 'req-9d99b1ed-bd27-4a4a-96d2-242ba8f97b31'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:15.346340 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-9d99b1ed-bd27-4a4a-96d2-242ba8f97b31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef579a625f6a46b39b8c21fabed3eb7c', 'request_id': 'req-9d99b1ed-bd27-4a4a-96d2-242ba8f97b31'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:15.359800 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-9d99b1ed-bd27-4a4a-96d2-242ba8f97b31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef579a625f6a46b39b8c21fabed3eb7c', 'request_id': 'req-9d99b1ed-bd27-4a4a-96d2-242ba8f97b31'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:15.410809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-9d99b1ed-bd27-4a4a-96d2-242ba8f97b31 admin admin] The token cache is being invalidate because project ef579a625f6a46b39b8c21fabed3eb7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:15.411101 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-9d99b1ed-bd27-4a4a-96d2-242ba8f97b31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:15.413987 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-9d99b1ed-bd27-4a4a-96d2-242ba8f97b31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:15.417215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 972/1963] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:15 2022] DELETE /identity/v3/projects/ef579a625f6a46b39b8c21fabed3eb7c => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:13:15.427549 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ad73e-512f-446d-a85f-a2d9cbaba99d None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:15.427826 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ad73e-512f-446d-a85f-a2d9cbaba99d None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:15.428337 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ad73e-512f-446d-a85f-a2d9cbaba99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:15.475823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-637ad73e-512f-446d-a85f-a2d9cbaba99d None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:15.477608 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-637ad73e-512f-446d-a85f-a2d9cbaba99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:15.502782 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 992/1964] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:15 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:15.524701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:15.597768 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b2964-7db3-4631-8738-4ca4fc445fad None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:15.600751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-169b2964-7db3-4631-8738-4ca4fc445fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:15.635627 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:15.645787 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 973/1965] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:15.652404 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e290e-03da-48e9-ac00-bcb025256b9d None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:15.652739 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e290e-03da-48e9-ac00-bcb025256b9d None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:15.653012 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e290e-03da-48e9-ac00-bcb025256b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:15.713080 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-4e3e290e-03da-48e9-ac00-bcb025256b9d None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:15.714851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-4e3e290e-03da-48e9-ac00-bcb025256b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:15.741199 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 974/1966] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:15.743643 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e290e-03da-48e9-ac00-bcb025256b9d None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:15.775280 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b2964-7db3-4631-8738-4ca4fc445fad None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:15.781364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fdbc3d-690f-4e46-b23c-3ad3269299bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:15.782005 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fdbc3d-690f-4e46-b23c-3ad3269299bc service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:15.782296 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fdbc3d-690f-4e46-b23c-3ad3269299bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:15.782995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fdbc3d-690f-4e46-b23c-3ad3269299bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:15.791450 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e290e-03da-48e9-ac00-bcb025256b9d None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:15.806249 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccacb7bc-c859-4d73-9ef2-60f1502102a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:15.807309 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacb7bc-c859-4d73-9ef2-60f1502102a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:15.808311 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacb7bc-c859-4d73-9ef2-60f1502102a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:15.808809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacb7bc-c859-4d73-9ef2-60f1502102a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:15.809763 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccacb7bc-c859-4d73-9ef2-60f1502102a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:13:15.810232 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccacb7bc-c859-4d73-9ef2-60f1502102a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:15.813219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccacb7bc-c859-4d73-9ef2-60f1502102a5 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:15.837717 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fdbc3d-690f-4e46-b23c-3ad3269299bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:15.841963 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fdbc3d-690f-4e46-b23c-3ad3269299bc service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:15.853486 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 975/1967] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:13:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:13:15.856579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccacb7bc-c859-4d73-9ef2-60f1502102a5 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:15.873636 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccacb7bc-c859-4d73-9ef2-60f1502102a5 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:15.879487 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 993/1968] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:15 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:15.880422 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607fa7e9-288e-4b0c-9af9-674cc8023150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:15.881536 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fa7e9-288e-4b0c-9af9-674cc8023150 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:15.881763 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fa7e9-288e-4b0c-9af9-674cc8023150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:15.882135 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fa7e9-288e-4b0c-9af9-674cc8023150 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:15.882901 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607fa7e9-288e-4b0c-9af9-674cc8023150 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:15.885931 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607fa7e9-288e-4b0c-9af9-674cc8023150 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:15.935193 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 976/1969] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:13:15 2022] POST /identity/v3/projects => generated 371 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:15.939303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fdbc3d-690f-4e46-b23c-3ad3269299bc service cinder] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:15.953879 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fdbc3d-690f-4e46-b23c-3ad3269299bc service cinder] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:16.009636 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607fa7e9-288e-4b0c-9af9-674cc8023150 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:16.012066 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-607fa7e9-288e-4b0c-9af9-674cc8023150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:16.063411 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fde929-892f-4fd7-bc22-7b9d5b5f110a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:16.064980 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fde929-892f-4fd7-bc22-7b9d5b5f110a admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:16.065390 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fde929-892f-4fd7-bc22-7b9d5b5f110a admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:16.065753 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fde929-892f-4fd7-bc22-7b9d5b5f110a admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:16.066623 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fde929-892f-4fd7-bc22-7b9d5b5f110a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:16.070838 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fde929-892f-4fd7-bc22-7b9d5b5f110a admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:16.093795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607fa7e9-288e-4b0c-9af9-674cc8023150 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:16.097650 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1a245-ed20-481f-b85f-b023fd4daf75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:16.098404 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1a245-ed20-481f-b85f-b023fd4daf75 service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:16.098685 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1a245-ed20-481f-b85f-b023fd4daf75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:16.098943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1a245-ed20-481f-b85f-b023fd4daf75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:16.100924 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-90b1a245-ed20-481f-b85f-b023fd4daf75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:16.117165 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 994/1970] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:13:15 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:16.119059 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fde929-892f-4fd7-bc22-7b9d5b5f110a admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:16.129817 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fde929-892f-4fd7-bc22-7b9d5b5f110a admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:16.135296 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ae286-5e4e-4cec-9fd3-5252d97a1644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:16.135934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ae286-5e4e-4cec-9fd3-5252d97a1644 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:16.136215 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ae286-5e4e-4cec-9fd3-5252d97a1644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:16.136481 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ae286-5e4e-4cec-9fd3-5252d97a1644 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:16.137662 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ae286-5e4e-4cec-9fd3-5252d97a1644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:16.138997 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ae286-5e4e-4cec-9fd3-5252d97a1644 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:16.150053 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 995/1971] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:13:16.154165 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ae286-5e4e-4cec-9fd3-5252d97a1644 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:16.164369 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ae286-5e4e-4cec-9fd3-5252d97a1644 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:16.166837 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b1a245-ed20-481f-b85f-b023fd4daf75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:16.168295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b1a245-ed20-481f-b85f-b023fd4daf75 service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:16.169956 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd9d34-96d7-445f-902e-56543e4d4c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:16.170591 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd9d34-96d7-445f-902e-56543e4d4c66 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:16.170866 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd9d34-96d7-445f-902e-56543e4d4c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:16.171136 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd9d34-96d7-445f-902e-56543e4d4c66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:16.171780 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd9d34-96d7-445f-902e-56543e4d4c66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:16.173480 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd9d34-96d7-445f-902e-56543e4d4c66 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:16.179320 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 977/1972] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:13:16.186026 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 996/1973] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:16.190181 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1a245-ed20-481f-b85f-b023fd4daf75 service cinder] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:16.205444 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1a245-ed20-481f-b85f-b023fd4daf75 service cinder] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:16.210917 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaebc1dd-562e-4742-99be-29b289d37b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:16.211797 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebc1dd-562e-4742-99be-29b289d37b3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:16.212146 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebc1dd-562e-4742-99be-29b289d37b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:16.212489 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebc1dd-562e-4742-99be-29b289d37b3e admin admin] PATH_INFO: `/v3/projects/1fffef8a1e8c4596bbfdbed900c1208d/users/c01e841877c944d7903a901c8ac24273/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:16.233321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaebc1dd-562e-4742-99be-29b289d37b3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fffef8a1e8c4596bbfdbed900c1208d, user_id=c01e841877c944d7903a901c8ac24273, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:16.236461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaebc1dd-562e-4742-99be-29b289d37b3e admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:16.252153 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 978/1974] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:16 2022] PUT /identity/v3/projects/1fffef8a1e8c4596bbfdbed900c1208d/users/c01e841877c944d7903a901c8ac24273/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:16.267449 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5ed2b-b9f3-469e-96a1-7859ce094bb4 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:16.267665 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5ed2b-b9f3-469e-96a1-7859ce094bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:16.267913 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5ed2b-b9f3-469e-96a1-7859ce094bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:16.299189 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-08c5ed2b-b9f3-469e-96a1-7859ce094bb4 None None] MFA Rules not processed for user `c01e841877c944d7903a901c8ac24273`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:16.300548 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-08c5ed2b-b9f3-469e-96a1-7859ce094bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:16.403418 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 997/1975] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:16.417775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaebc1dd-562e-4742-99be-29b289d37b3e admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:16.450292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:16.511791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaebc1dd-562e-4742-99be-29b289d37b3e admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:16.516029 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e5a47-cbc7-4091-b6d3-007d27a58b33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:16.517027 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e5a47-cbc7-4091-b6d3-007d27a58b33 service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:16.517545 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e5a47-cbc7-4091-b6d3-007d27a58b33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:16.517760 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e5a47-cbc7-4091-b6d3-007d27a58b33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:16.534602 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e5a47-cbc7-4091-b6d3-007d27a58b33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:16.537568 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e5a47-cbc7-4091-b6d3-007d27a58b33 service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:16.566610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 979/1976] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:16.602740 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c5ed2b-b9f3-469e-96a1-7859ce094bb4 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:16.622196 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:16.684906 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c5ed2b-b9f3-469e-96a1-7859ce094bb4 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:16.689497 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095746b4-b324-4aa4-b576-fa9dcd913bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:16.690166 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095746b4-b324-4aa4-b576-fa9dcd913bbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:16.690435 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095746b4-b324-4aa4-b576-fa9dcd913bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:16.690702 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095746b4-b324-4aa4-b576-fa9dcd913bbd admin admin] PATH_INFO: `/v3/users/c01e841877c944d7903a901c8ac24273` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:16.691855 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095746b4-b324-4aa4-b576-fa9dcd913bbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c01e841877c944d7903a901c8ac24273)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:16.694468 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095746b4-b324-4aa4-b576-fa9dcd913bbd admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:16.758786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-095746b4-b324-4aa4-b576-fa9dcd913bbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c01e841877c944d7903a901c8ac24273'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:16.767632 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-095746b4-b324-4aa4-b576-fa9dcd913bbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c01e841877c944d7903a901c8ac24273'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:16.784007 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-095746b4-b324-4aa4-b576-fa9dcd913bbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c01e841877c944d7903a901c8ac24273'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:16.785503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-095746b4-b324-4aa4-b576-fa9dcd913bbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c01e841877c944d7903a901c8ac24273'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:16.796556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-095746b4-b324-4aa4-b576-fa9dcd913bbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c01e841877c944d7903a901c8ac24273'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:16.803677 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e5a47-cbc7-4091-b6d3-007d27a58b33 service cinder] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:16.804169 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 998/1977] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:16 2022] DELETE /identity/v3/users/c01e841877c944d7903a901c8ac24273 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:13:16.805428 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-538e5a47-cbc7-4091-b6d3-007d27a58b33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:16.808589 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22dcaf-9c4a-4e50-9a82-9f838fa6808c None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:16.808937 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22dcaf-9c4a-4e50-9a82-9f838fa6808c None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:16.809288 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22dcaf-9c4a-4e50-9a82-9f838fa6808c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:16.823395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e5a47-cbc7-4091-b6d3-007d27a58b33 service cinder] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:16.831469 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-ae22dcaf-9c4a-4e50-9a82-9f838fa6808c None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:16.832690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-ae22dcaf-9c4a-4e50-9a82-9f838fa6808c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:16.899487 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089edcf-38ab-4bf0-ab57-ad651ab7c41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:16.900158 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089edcf-38ab-4bf0-ab57-ad651ab7c41f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:16.900818 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089edcf-38ab-4bf0-ab57-ad651ab7c41f admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:16.901335 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089edcf-38ab-4bf0-ab57-ad651ab7c41f admin admin] PATH_INFO: `/v3/users/0f2c82c962454acc90fe051746b6fceb` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:16.903284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089edcf-38ab-4bf0-ab57-ad651ab7c41f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f2c82c962454acc90fe051746b6fceb)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:16.905900 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089edcf-38ab-4bf0-ab57-ad651ab7c41f admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:16.931139 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 999/1978] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:13:16.942944 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22dcaf-9c4a-4e50-9a82-9f838fa6808c None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:16.944414 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-ae22dcaf-9c4a-4e50-9a82-9f838fa6808c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:16.962003 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-9089edcf-38ab-4bf0-ab57-ad651ab7c41f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f2c82c962454acc90fe051746b6fceb'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:16.974284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-9089edcf-38ab-4bf0-ab57-ad651ab7c41f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f2c82c962454acc90fe051746b6fceb'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:16.986470 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:16.995985 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-9089edcf-38ab-4bf0-ab57-ad651ab7c41f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f2c82c962454acc90fe051746b6fceb'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:16.997327 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-9089edcf-38ab-4bf0-ab57-ad651ab7c41f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f2c82c962454acc90fe051746b6fceb'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:17.007195 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-9089edcf-38ab-4bf0-ab57-ad651ab7c41f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f2c82c962454acc90fe051746b6fceb'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:17.014356 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22dcaf-9c4a-4e50-9a82-9f838fa6808c None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:17.017181 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 980/1979] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:16 2022] DELETE /identity/v3/users/0f2c82c962454acc90fe051746b6fceb => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:13:17.029455 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cb926-fa8d-4ef5-ba49-cf51b8661c2f None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:17.029455 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cb926-fa8d-4ef5-ba49-cf51b8661c2f None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:17.029455 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cb926-fa8d-4ef5-ba49-cf51b8661c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:17.043125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-842cb926-fa8d-4ef5-ba49-cf51b8661c2f None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:17.044527 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-842cb926-fa8d-4ef5-ba49-cf51b8661c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:17.084948 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadeaeb9-d163-4b6d-b30a-cace91327cde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:17.085685 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadeaeb9-d163-4b6d-b30a-cace91327cde service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:17.086112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadeaeb9-d163-4b6d-b30a-cace91327cde service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:17.086446 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadeaeb9-d163-4b6d-b30a-cace91327cde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:17.087978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:17.091062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-dadeaeb9-d163-4b6d-b30a-cace91327cde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:17.134859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 981/1980] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:13:17.141816 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842cb926-fa8d-4ef5-ba49-cf51b8661c2f None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:17.143387 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-842cb926-fa8d-4ef5-ba49-cf51b8661c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:17.161825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842cb926-fa8d-4ef5-ba49-cf51b8661c2f None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:17.166243 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a66f8-4f16-45c2-bfd1-8cd012251baa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:17.166907 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a66f8-4f16-45c2-bfd1-8cd012251baa service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:17.167188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a66f8-4f16-45c2-bfd1-8cd012251baa service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:17.167453 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a66f8-4f16-45c2-bfd1-8cd012251baa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:17.183018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a66f8-4f16-45c2-bfd1-8cd012251baa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:17.185251 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a66f8-4f16-45c2-bfd1-8cd012251baa service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:17.190781 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadeaeb9-d163-4b6d-b30a-cace91327cde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:17.193529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadeaeb9-d163-4b6d-b30a-cace91327cde service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:17.200512 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 982/1981] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:13:17.209671 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1000/1982] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:17.558733 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a66f8-4f16-45c2-bfd1-8cd012251baa service neutron] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:17.560270 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-a00a66f8-4f16-45c2-bfd1-8cd012251baa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:17.576775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a66f8-4f16-45c2-bfd1-8cd012251baa service neutron] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:17.582637 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728cbc0-219b-48c4-8e44-028646113dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:17.583544 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728cbc0-219b-48c4-8e44-028646113dc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:17.583930 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728cbc0-219b-48c4-8e44-028646113dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:17.584079 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728cbc0-219b-48c4-8e44-028646113dc4 admin admin] PATH_INFO: `/v3/projects/3b37cd218e1f4a8181be9f8eac809535` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:17.585476 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1728cbc0-219b-48c4-8e44-028646113dc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b37cd218e1f4a8181be9f8eac809535)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:17.587448 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1728cbc0-219b-48c4-8e44-028646113dc4 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:17.659832 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-1728cbc0-219b-48c4-8e44-028646113dc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b37cd218e1f4a8181be9f8eac809535', 'request_id': 'req-1728cbc0-219b-48c4-8e44-028646113dc4'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:17.669888 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-1728cbc0-219b-48c4-8e44-028646113dc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b37cd218e1f4a8181be9f8eac809535', 'request_id': 'req-1728cbc0-219b-48c4-8e44-028646113dc4'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:17.681411 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-1728cbc0-219b-48c4-8e44-028646113dc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b37cd218e1f4a8181be9f8eac809535', 'request_id': 'req-1728cbc0-219b-48c4-8e44-028646113dc4'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:17.702167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-1728cbc0-219b-48c4-8e44-028646113dc4 admin admin] The token cache is being invalidate because project 3b37cd218e1f4a8181be9f8eac809535 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:17.702167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-1728cbc0-219b-48c4-8e44-028646113dc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:17.709754 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-1728cbc0-219b-48c4-8e44-028646113dc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:17.711987 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 983/1983] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:17 2022] DELETE /identity/v3/projects/3b37cd218e1f4a8181be9f8eac809535 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:17.716993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadeaeb9-d163-4b6d-b30a-cace91327cde service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:17.740882 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadeaeb9-d163-4b6d-b30a-cace91327cde service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:17.849787 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f97a06-66e5-46b2-894b-e6175903d419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:17.850512 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f97a06-66e5-46b2-894b-e6175903d419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:17.850764 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f97a06-66e5-46b2-894b-e6175903d419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:17.851032 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f97a06-66e5-46b2-894b-e6175903d419 admin admin] PATH_INFO: `/v3/users/1e3628e18ff240ca80707e1ac238a6a2` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:17.852401 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f97a06-66e5-46b2-894b-e6175903d419 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e3628e18ff240ca80707e1ac238a6a2)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:17.854884 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728cbc0-219b-48c4-8e44-028646113dc4 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:17.855341 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f97a06-66e5-46b2-894b-e6175903d419 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:17.856919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-1728cbc0-219b-48c4-8e44-028646113dc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:17.886646 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728cbc0-219b-48c4-8e44-028646113dc4 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:17.895701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac23802f-dcd5-4046-b61e-4494a3bc9786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:17.896383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23802f-dcd5-4046-b61e-4494a3bc9786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:17.896654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23802f-dcd5-4046-b61e-4494a3bc9786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:17.897097 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23802f-dcd5-4046-b61e-4494a3bc9786 admin admin] PATH_INFO: `/v3/projects/1fffef8a1e8c4596bbfdbed900c1208d` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:17.898693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac23802f-dcd5-4046-b61e-4494a3bc9786 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fffef8a1e8c4596bbfdbed900c1208d)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:17.902981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac23802f-dcd5-4046-b61e-4494a3bc9786 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:17.946431 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:17.960915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-c9f97a06-66e5-46b2-894b-e6175903d419 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e3628e18ff240ca80707e1ac238a6a2'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:17.970955 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-c9f97a06-66e5-46b2-894b-e6175903d419 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e3628e18ff240ca80707e1ac238a6a2'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:18.003834 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-ac23802f-dcd5-4046-b61e-4494a3bc9786 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fffef8a1e8c4596bbfdbed900c1208d', 'request_id': 'req-ac23802f-dcd5-4046-b61e-4494a3bc9786'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:18.014617 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-c9f97a06-66e5-46b2-894b-e6175903d419 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e3628e18ff240ca80707e1ac238a6a2'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:18.015593 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-c9f97a06-66e5-46b2-894b-e6175903d419 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e3628e18ff240ca80707e1ac238a6a2'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:18.024750 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-ac23802f-dcd5-4046-b61e-4494a3bc9786 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fffef8a1e8c4596bbfdbed900c1208d', 'request_id': 'req-ac23802f-dcd5-4046-b61e-4494a3bc9786'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:18.028439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-c9f97a06-66e5-46b2-894b-e6175903d419 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e3628e18ff240ca80707e1ac238a6a2'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:18.034641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-ac23802f-dcd5-4046-b61e-4494a3bc9786 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fffef8a1e8c4596bbfdbed900c1208d', 'request_id': 'req-ac23802f-dcd5-4046-b61e-4494a3bc9786'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:18.036150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1001/1984] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:17 2022] DELETE /identity/v3/users/1e3628e18ff240ca80707e1ac238a6a2 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:18.050648 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-ac23802f-dcd5-4046-b61e-4494a3bc9786 admin admin] The token cache is being invalidate because project 1fffef8a1e8c4596bbfdbed900c1208d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:18.050954 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-ac23802f-dcd5-4046-b61e-4494a3bc9786 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:18.051680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-ac23802f-dcd5-4046-b61e-4494a3bc9786 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:18.057423 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 984/1985] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:17 2022] DELETE /identity/v3/projects/1fffef8a1e8c4596bbfdbed900c1208d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:13:18.063039 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e2b4a-d39f-46f5-a040-8c5666ec044b None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:18.063327 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e2b4a-d39f-46f5-a040-8c5666ec044b None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:18.063620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e2b4a-d39f-46f5-a040-8c5666ec044b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:18.088715 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-c21e2b4a-d39f-46f5-a040-8c5666ec044b None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:18.090471 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-c21e2b4a-d39f-46f5-a040-8c5666ec044b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:18.163622 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1002/1986] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:18.171892 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8552c-b723-4894-8d1e-2d5472e9c567 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:18.172173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8552c-b723-4894-8d1e-2d5472e9c567 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:18.172440 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8552c-b723-4894-8d1e-2d5472e9c567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:18.192677 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-dac8552c-b723-4894-8d1e-2d5472e9c567 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:18.194022 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-dac8552c-b723-4894-8d1e-2d5472e9c567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:18.207209 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 985/1987] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:18.208707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e2b4a-d39f-46f5-a040-8c5666ec044b None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:18.224769 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e2b4a-d39f-46f5-a040-8c5666ec044b None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:18.230525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd6837-33d7-4456-a06c-c75c5c185185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:18.231152 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd6837-33d7-4456-a06c-c75c5c185185 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:18.231425 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd6837-33d7-4456-a06c-c75c5c185185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:18.231687 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd6837-33d7-4456-a06c-c75c5c185185 admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:18.232351 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd6837-33d7-4456-a06c-c75c5c185185 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:13:18.232656 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd6837-33d7-4456-a06c-c75c5c185185 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:18.234309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd6837-33d7-4456-a06c-c75c5c185185 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:18.248862 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1003/1988] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:13:18.253073 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8552c-b723-4894-8d1e-2d5472e9c567 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:18.263670 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8552c-b723-4894-8d1e-2d5472e9c567 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:18.268397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a9b71-acc0-4231-957f-2e8207a2b94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:18.269166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a9b71-acc0-4231-957f-2e8207a2b94b admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:18.269467 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a9b71-acc0-4231-957f-2e8207a2b94b admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:18.269738 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a9b71-acc0-4231-957f-2e8207a2b94b admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:18.270365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783a9b71-acc0-4231-957f-2e8207a2b94b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:18.272385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783a9b71-acc0-4231-957f-2e8207a2b94b admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:18.296880 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:18.309710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 986/1989] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:13:18 2022] POST /identity/v3/projects => generated 391 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:13:18.310385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd6837-33d7-4456-a06c-c75c5c185185 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:18.322726 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd6837-33d7-4456-a06c-c75c5c185185 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:18.375807 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a9b71-acc0-4231-957f-2e8207a2b94b admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:18.380427 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-783a9b71-acc0-4231-957f-2e8207a2b94b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:18.406373 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ffdf9-e998-4f15-8a86-63140a1d260d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:18.406373 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ffdf9-e998-4f15-8a86-63140a1d260d admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:18.406373 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ffdf9-e998-4f15-8a86-63140a1d260d admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:18.406743 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ffdf9-e998-4f15-8a86-63140a1d260d admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:18.407410 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901ffdf9-e998-4f15-8a86-63140a1d260d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:18.414666 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901ffdf9-e998-4f15-8a86-63140a1d260d admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:18.440260 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1004/1990] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:13:18 2022] POST /identity/v3/users => generated 296 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:18.444658 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ffdf9-e998-4f15-8a86-63140a1d260d admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:18.456764 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ffdf9-e998-4f15-8a86-63140a1d260d admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:18.462303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21c50f-6e5d-4a5e-bcb1-e34554c2ea73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:18.462743 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21c50f-6e5d-4a5e-bcb1-e34554c2ea73 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:18.463080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21c50f-6e5d-4a5e-bcb1-e34554c2ea73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:18.463409 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21c50f-6e5d-4a5e-bcb1-e34554c2ea73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:18.464148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21c50f-6e5d-4a5e-bcb1-e34554c2ea73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:18.465855 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21c50f-6e5d-4a5e-bcb1-e34554c2ea73 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:18.472258 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a9b71-acc0-4231-957f-2e8207a2b94b admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:18.477237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452d993-fa2c-4e0e-ba44-4d4b38c93157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:18.477908 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452d993-fa2c-4e0e-ba44-4d4b38c93157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:18.478184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452d993-fa2c-4e0e-ba44-4d4b38c93157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:18.478449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452d993-fa2c-4e0e-ba44-4d4b38c93157 admin admin] PATH_INFO: `/v3/projects/c56fc966b38646c6aaf97b353ca76190` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:18.479593 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1005/1991] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:13:18.480302 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5452d993-fa2c-4e0e-ba44-4d4b38c93157 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c56fc966b38646c6aaf97b353ca76190)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:18.482362 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5452d993-fa2c-4e0e-ba44-4d4b38c93157 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:18.483229 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21c50f-6e5d-4a5e-bcb1-e34554c2ea73 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:18.494881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21c50f-6e5d-4a5e-bcb1-e34554c2ea73 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:18.500278 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56b5c3-105c-4e41-b9c2-ec8027ec20a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:18.500887 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56b5c3-105c-4e41-b9c2-ec8027ec20a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:18.501173 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56b5c3-105c-4e41-b9c2-ec8027ec20a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:18.501445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56b5c3-105c-4e41-b9c2-ec8027ec20a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:18.502084 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56b5c3-105c-4e41-b9c2-ec8027ec20a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:18.503871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56b5c3-105c-4e41-b9c2-ec8027ec20a0 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:18.518995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1006/1992] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:18.523681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56b5c3-105c-4e41-b9c2-ec8027ec20a0 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:18.533901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56b5c3-105c-4e41-b9c2-ec8027ec20a0 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:18.539242 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-5452d993-fa2c-4e0e-ba44-4d4b38c93157 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c56fc966b38646c6aaf97b353ca76190', 'request_id': 'req-5452d993-fa2c-4e0e-ba44-4d4b38c93157'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:18.543283 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:18.549612 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-5452d993-fa2c-4e0e-ba44-4d4b38c93157 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c56fc966b38646c6aaf97b353ca76190', 'request_id': 'req-5452d993-fa2c-4e0e-ba44-4d4b38c93157'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:18.557679 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-5452d993-fa2c-4e0e-ba44-4d4b38c93157 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c56fc966b38646c6aaf97b353ca76190', 'request_id': 'req-5452d993-fa2c-4e0e-ba44-4d4b38c93157'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:18.570030 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-5452d993-fa2c-4e0e-ba44-4d4b38c93157 admin admin] The token cache is being invalidate because project c56fc966b38646c6aaf97b353ca76190 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:18.570321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-5452d993-fa2c-4e0e-ba44-4d4b38c93157 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:18.571095 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-5452d993-fa2c-4e0e-ba44-4d4b38c93157 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:18.572804 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 987/1993] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:18 2022] DELETE /identity/v3/projects/c56fc966b38646c6aaf97b353ca76190 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:13:18.578260 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f2900-a42e-4a5e-ace7-515d943b34b3 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:18.578545 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f2900-a42e-4a5e-ace7-515d943b34b3 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:18.578812 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f2900-a42e-4a5e-ace7-515d943b34b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:18.598120 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ae6a6-cc5c-40b8-9b22-f2788428c3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:18.598843 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ae6a6-cc5c-40b8-9b22-f2788428c3ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:18.599085 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ae6a6-cc5c-40b8-9b22-f2788428c3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:18.599360 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ae6a6-cc5c-40b8-9b22-f2788428c3ef admin admin] PATH_INFO: `/v3/projects/e007facf43ab4107a24d68bfeed2e716/users/8f684613fb624c1c9ae85d34c3c2239a/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:18.599722 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-3a8f2900-a42e-4a5e-ace7-515d943b34b3 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:18.601074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-3a8f2900-a42e-4a5e-ace7-515d943b34b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:18.609306 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 988/1994] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:13:18.614978 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798ae6a6-cc5c-40b8-9b22-f2788428c3ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=e007facf43ab4107a24d68bfeed2e716, user_id=8f684613fb624c1c9ae85d34c3c2239a, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:18.617008 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a02b4-fafa-439f-b6db-730e1ab4c4bf None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:18.617329 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a02b4-fafa-439f-b6db-730e1ab4c4bf None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:18.617599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a02b4-fafa-439f-b6db-730e1ab4c4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:18.617830 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798ae6a6-cc5c-40b8-9b22-f2788428c3ef admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:18.632580 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1007/1995] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:18 2022] PUT /identity/v3/projects/e007facf43ab4107a24d68bfeed2e716/users/8f684613fb624c1c9ae85d34c3c2239a/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:13:18.637802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-2e9a02b4-fafa-439f-b6db-730e1ab4c4bf None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:18.639031 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-2e9a02b4-fafa-439f-b6db-730e1ab4c4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:18.643212 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c71b3d-4343-48b2-91fa-8922c42a3fe1 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:18.643556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c71b3d-4343-48b2-91fa-8922c42a3fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:18.643850 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c71b3d-4343-48b2-91fa-8922c42a3fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:18.671860 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-73c71b3d-4343-48b2-91fa-8922c42a3fe1 None None] MFA Rules not processed for user `8f684613fb624c1c9ae85d34c3c2239a`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:18.673062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-73c71b3d-4343-48b2-91fa-8922c42a3fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:18.701838 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 989/1996] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:13:18.706306 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a02b4-fafa-439f-b6db-730e1ab4c4bf None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:18.715246 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a02b4-fafa-439f-b6db-730e1ab4c4bf None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:18.719936 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c40b7e-d215-4fcf-99b9-7262fbdaba10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:18.720545 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c40b7e-d215-4fcf-99b9-7262fbdaba10 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:18.720821 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c40b7e-d215-4fcf-99b9-7262fbdaba10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:18.721097 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c40b7e-d215-4fcf-99b9-7262fbdaba10 admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:18.721792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c40b7e-d215-4fcf-99b9-7262fbdaba10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:13:18.722097 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c40b7e-d215-4fcf-99b9-7262fbdaba10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:18.723710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c40b7e-d215-4fcf-99b9-7262fbdaba10 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:18.739308 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 990/1997] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:13:18.740968 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1008/1998] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:18.741235 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c40b7e-d215-4fcf-99b9-7262fbdaba10 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:18.752361 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c40b7e-d215-4fcf-99b9-7262fbdaba10 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:18.753535 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c71b3d-4343-48b2-91fa-8922c42a3fe1 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:18.754907 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-73c71b3d-4343-48b2-91fa-8922c42a3fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:18.757823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486509b-7d19-422b-a725-7eba0f315e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:18.758462 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486509b-7d19-422b-a725-7eba0f315e0a admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:18.758742 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486509b-7d19-422b-a725-7eba0f315e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:18.759017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486509b-7d19-422b-a725-7eba0f315e0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:18.760168 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5486509b-7d19-422b-a725-7eba0f315e0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:18.763425 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5486509b-7d19-422b-a725-7eba0f315e0a admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:18.775941 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:18.799135 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 991/1999] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:13:18 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:18.804477 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486509b-7d19-422b-a725-7eba0f315e0a admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:18.830122 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c71b3d-4343-48b2-91fa-8922c42a3fe1 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:18.884355 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:18.898651 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f59bbf-dcd3-4fd7-8156-6a278262e592 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:18.900354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f59bbf-dcd3-4fd7-8156-6a278262e592 service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:18.900664 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f59bbf-dcd3-4fd7-8156-6a278262e592 service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:18.900936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f59bbf-dcd3-4fd7-8156-6a278262e592 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:18.901207 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486509b-7d19-422b-a725-7eba0f315e0a admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:18.908292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f261d-a8fe-494b-b320-b74172e13ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:18.908622 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f261d-a8fe-494b-b320-b74172e13ebd admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:18.908840 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f261d-a8fe-494b-b320-b74172e13ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:18.909111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f261d-a8fe-494b-b320-b74172e13ebd admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:18.910037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f261d-a8fe-494b-b320-b74172e13ebd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:18.913058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f261d-a8fe-494b-b320-b74172e13ebd admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:18.938966 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 992/2000] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:13:18 2022] POST /identity/v3/users => generated 288 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:13:18.945473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f261d-a8fe-494b-b320-b74172e13ebd admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:18.958761 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f261d-a8fe-494b-b320-b74172e13ebd admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:18.965155 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de90de-8296-4d47-935a-3562ba1e7e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:18.965857 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de90de-8296-4d47-935a-3562ba1e7e86 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:18.966137 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de90de-8296-4d47-935a-3562ba1e7e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:18.966411 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de90de-8296-4d47-935a-3562ba1e7e86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:18.967076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de90de-8296-4d47-935a-3562ba1e7e86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:18.968725 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de90de-8296-4d47-935a-3562ba1e7e86 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:18.981852 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 993/2001] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:13:18.986438 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de90de-8296-4d47-935a-3562ba1e7e86 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:18.998356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de90de-8296-4d47-935a-3562ba1e7e86 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:19.009109 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d70b4c4-1ff0-4841-9b24-463b0e7ff7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:19.009774 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70b4c4-1ff0-4841-9b24-463b0e7ff7a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:19.010110 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70b4c4-1ff0-4841-9b24-463b0e7ff7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:19.012018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f59bbf-dcd3-4fd7-8156-6a278262e592 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:19.012950 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70b4c4-1ff0-4841-9b24-463b0e7ff7a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:19.014162 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d70b4c4-1ff0-4841-9b24-463b0e7ff7a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:19.014306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f59bbf-dcd3-4fd7-8156-6a278262e592 service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:19.015772 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d70b4c4-1ff0-4841-9b24-463b0e7ff7a0 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:19.028076 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1009/2002] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:18 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:13:19.038899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 994/2003] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:13:19.044611 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f59bbf-dcd3-4fd7-8156-6a278262e592 service cinder] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:19.055828 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f59bbf-dcd3-4fd7-8156-6a278262e592 service cinder] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:19.062513 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72619e-cf71-4cc1-a5fe-96f025665c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:19.063354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72619e-cf71-4cc1-a5fe-96f025665c2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:19.063739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72619e-cf71-4cc1-a5fe-96f025665c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:19.063889 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72619e-cf71-4cc1-a5fe-96f025665c2e admin admin] PATH_INFO: `/v3/projects/7010b191373145608fe85e0e89085a63/users/259bbefd7cec436188717c99e6b340e5/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:19.092540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c72619e-cf71-4cc1-a5fe-96f025665c2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7010b191373145608fe85e0e89085a63, user_id=259bbefd7cec436188717c99e6b340e5, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:19.096098 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c72619e-cf71-4cc1-a5fe-96f025665c2e admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:19.117630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1010/2004] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:19 2022] PUT /identity/v3/projects/7010b191373145608fe85e0e89085a63/users/259bbefd7cec436188717c99e6b340e5/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:19.129704 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa272dd-e1ac-4bfc-99e9-c56bad6ba859 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:19.130021 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa272dd-e1ac-4bfc-99e9-c56bad6ba859 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:19.130375 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa272dd-e1ac-4bfc-99e9-c56bad6ba859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:19.164318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-5fa272dd-e1ac-4bfc-99e9-c56bad6ba859 None None] MFA Rules not processed for user `259bbefd7cec436188717c99e6b340e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:19.165700 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-5fa272dd-e1ac-4bfc-99e9-c56bad6ba859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:19.270052 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 995/2005] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:19.270512 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72619e-cf71-4cc1-a5fe-96f025665c2e admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:19.289804 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa272dd-e1ac-4bfc-99e9-c56bad6ba859 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:19.291344 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-5fa272dd-e1ac-4bfc-99e9-c56bad6ba859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:19.379199 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:19.399474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72619e-cf71-4cc1-a5fe-96f025665c2e admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:19.404658 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8544f17-f2d6-44e2-9359-61942e18d68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:19.405320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8544f17-f2d6-44e2-9359-61942e18d68e admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:19.405459 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa272dd-e1ac-4bfc-99e9-c56bad6ba859 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:19.405602 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8544f17-f2d6-44e2-9359-61942e18d68e admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:19.405861 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8544f17-f2d6-44e2-9359-61942e18d68e admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:19.406714 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8544f17-f2d6-44e2-9359-61942e18d68e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:19.409169 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8544f17-f2d6-44e2-9359-61942e18d68e admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:19.411160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6295fb4-cd62-493c-b3bc-60b358652c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:19.413830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6295fb4-cd62-493c-b3bc-60b358652c23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:19.414112 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6295fb4-cd62-493c-b3bc-60b358652c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:19.414423 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6295fb4-cd62-493c-b3bc-60b358652c23 admin admin] PATH_INFO: `/v3/users/a380060812584f83b0655eaa67badab4` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:19.415646 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6295fb4-cd62-493c-b3bc-60b358652c23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a380060812584f83b0655eaa67badab4)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:19.417762 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6295fb4-cd62-493c-b3bc-60b358652c23 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:19.455408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1011/2006] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:13:19 2022] POST /identity/v3/projects => generated 377 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:19.459410 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8544f17-f2d6-44e2-9359-61942e18d68e admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:19.513262 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d6295fb4-cd62-493c-b3bc-60b358652c23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a380060812584f83b0655eaa67badab4'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:19.535541 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d6295fb4-cd62-493c-b3bc-60b358652c23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a380060812584f83b0655eaa67badab4'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:19.569768 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d6295fb4-cd62-493c-b3bc-60b358652c23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a380060812584f83b0655eaa67badab4'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:19.569768 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d6295fb4-cd62-493c-b3bc-60b358652c23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a380060812584f83b0655eaa67badab4'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:19.597791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d6295fb4-cd62-493c-b3bc-60b358652c23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a380060812584f83b0655eaa67badab4'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:19.612576 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 996/2007] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:19 2022] DELETE /identity/v3/users/a380060812584f83b0655eaa67badab4 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:19.619858 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fedd09-ccf6-4eec-9ca9-ff9658a9f199 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:19.623885 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fedd09-ccf6-4eec-9ca9-ff9658a9f199 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:19.628780 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:19.631646 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fedd09-ccf6-4eec-9ca9-ff9658a9f199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:19.635218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8544f17-f2d6-44e2-9359-61942e18d68e admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:19.688113 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-99fedd09-ccf6-4eec-9ca9-ff9658a9f199 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:19.690501 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-99fedd09-ccf6-4eec-9ca9-ff9658a9f199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:19.717032 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad02fa4-981e-45d3-8038-4830c3b9a422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:19.717818 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad02fa4-981e-45d3-8038-4830c3b9a422 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:19.718159 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad02fa4-981e-45d3-8038-4830c3b9a422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:19.718539 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad02fa4-981e-45d3-8038-4830c3b9a422 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:19.719314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad02fa4-981e-45d3-8038-4830c3b9a422 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:19.721438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad02fa4-981e-45d3-8038-4830c3b9a422 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:19.760199 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1012/2008] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:13:19 2022] POST /identity/v3/users => generated 289 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:13:19.766581 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad02fa4-981e-45d3-8038-4830c3b9a422 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:19.770450 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-6ad02fa4-981e-45d3-8038-4830c3b9a422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:19.789288 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 997/2009] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:19.797960 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad02fa4-981e-45d3-8038-4830c3b9a422 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:19.806266 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b135a087-f574-46ea-82ab-355ad3412725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:19.807447 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b135a087-f574-46ea-82ab-355ad3412725 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:19.807885 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b135a087-f574-46ea-82ab-355ad3412725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:19.808353 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b135a087-f574-46ea-82ab-355ad3412725 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:19.809562 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b135a087-f574-46ea-82ab-355ad3412725 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:19.812554 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b135a087-f574-46ea-82ab-355ad3412725 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:19.837225 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1013/2010] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:13:19.838774 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fedd09-ccf6-4eec-9ca9-ff9658a9f199 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:19.845300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b135a087-f574-46ea-82ab-355ad3412725 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:19.848590 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-b135a087-f574-46ea-82ab-355ad3412725 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:19.851102 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fedd09-ccf6-4eec-9ca9-ff9658a9f199 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:19.856479 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2950900c-1447-4199-b044-5c77059c7736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:19.857190 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950900c-1447-4199-b044-5c77059c7736 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:19.857477 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950900c-1447-4199-b044-5c77059c7736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:19.857746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950900c-1447-4199-b044-5c77059c7736 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:19.858401 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2950900c-1447-4199-b044-5c77059c7736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:19.861266 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2950900c-1447-4199-b044-5c77059c7736 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:19.879339 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 998/2011] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:13:19.886413 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2950900c-1447-4199-b044-5c77059c7736 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:19.899730 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2950900c-1447-4199-b044-5c77059c7736 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:19.907384 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b10bd-86cc-4c8f-91a4-0775e25838f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:19.908252 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b10bd-86cc-4c8f-91a4-0775e25838f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:19.917606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b10bd-86cc-4c8f-91a4-0775e25838f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:19.917887 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b10bd-86cc-4c8f-91a4-0775e25838f6 admin admin] PATH_INFO: `/v3/projects/1d2fc087f1574e24869bc01f1f49f064/users/a0ee273924b24ef39f10e75ebeed6098/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:19.941161 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508b10bd-86cc-4c8f-91a4-0775e25838f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d2fc087f1574e24869bc01f1f49f064, user_id=a0ee273924b24ef39f10e75ebeed6098, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:19.946460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508b10bd-86cc-4c8f-91a4-0775e25838f6 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:19.976040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b135a087-f574-46ea-82ab-355ad3412725 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:19.981351 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 999/2012] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:19 2022] PUT /identity/v3/projects/1d2fc087f1574e24869bc01f1f49f064/users/a0ee273924b24ef39f10e75ebeed6098/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:19.994701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cfc49-836d-426c-8ad5-aa8ee8b75c16 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:19.995349 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cfc49-836d-426c-8ad5-aa8ee8b75c16 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:19.995888 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cfc49-836d-426c-8ad5-aa8ee8b75c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:20.077854 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-272cfc49-836d-426c-8ad5-aa8ee8b75c16 None None] MFA Rules not processed for user `a0ee273924b24ef39f10e75ebeed6098`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:20.080042 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-272cfc49-836d-426c-8ad5-aa8ee8b75c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:20.112620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5987b-3e61-43b9-88fb-b4a124160796 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:20.114342 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5987b-3e61-43b9-88fb-b4a124160796 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:20.115003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5987b-3e61-43b9-88fb-b4a124160796 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:20.116090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5987b-3e61-43b9-88fb-b4a124160796 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:20.122490 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:20.218980 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:20.235383 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5987b-3e61-43b9-88fb-b4a124160796 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:20.237297 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5987b-3e61-43b9-88fb-b4a124160796 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:20.253510 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1014/2013] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:20.265331 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1000/2014] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:13:20.269992 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5987b-3e61-43b9-88fb-b4a124160796 service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:20.290741 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5987b-3e61-43b9-88fb-b4a124160796 service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:20.297339 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec50b7-a1a8-4144-8e33-f126ec06133e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:20.298188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec50b7-a1a8-4144-8e33-f126ec06133e admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:20.298635 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec50b7-a1a8-4144-8e33-f126ec06133e admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:20.299057 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec50b7-a1a8-4144-8e33-f126ec06133e admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:20.299989 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec50b7-a1a8-4144-8e33-f126ec06133e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:20.302380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec50b7-a1a8-4144-8e33-f126ec06133e admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:20.337320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1015/2015] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:13:20 2022] POST /identity/v3/projects => generated 377 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:13:20.342151 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272cfc49-836d-426c-8ad5-aa8ee8b75c16 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:20.358224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272cfc49-836d-426c-8ad5-aa8ee8b75c16 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:20.432974 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57397380-f441-4ccd-ad87-d6d0071d090f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:20.433640 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57397380-f441-4ccd-ad87-d6d0071d090f admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:20.433920 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57397380-f441-4ccd-ad87-d6d0071d090f admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:20.434197 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57397380-f441-4ccd-ad87-d6d0071d090f admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:20.435008 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57397380-f441-4ccd-ad87-d6d0071d090f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:20.437035 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57397380-f441-4ccd-ad87-d6d0071d090f admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:20.466383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1001/2016] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:13:20 2022] POST /identity/v3/users => generated 295 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:20.470521 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec50b7-a1a8-4144-8e33-f126ec06133e admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:20.484918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec50b7-a1a8-4144-8e33-f126ec06133e admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:20.492341 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa517c-4152-469a-949e-8cdef8606d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:20.493339 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa517c-4152-469a-949e-8cdef8606d40 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:20.493719 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa517c-4152-469a-949e-8cdef8606d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:20.494089 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa517c-4152-469a-949e-8cdef8606d40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:20.494991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa517c-4152-469a-949e-8cdef8606d40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:20.496724 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa517c-4152-469a-949e-8cdef8606d40 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:20.512690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1016/2017] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:13:20.517211 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57397380-f441-4ccd-ad87-d6d0071d090f admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:20.534014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57397380-f441-4ccd-ad87-d6d0071d090f admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:20.542320 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4497039-f3dd-467c-97b2-890e35f6c670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:20.542944 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4497039-f3dd-467c-97b2-890e35f6c670 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:20.543229 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4497039-f3dd-467c-97b2-890e35f6c670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:20.543600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4497039-f3dd-467c-97b2-890e35f6c670 admin admin] PATH_INFO: `/v3/projects/881128a82b8d4fb1a0b500801ee5397e/users/bb7efb83d5e645249213bb11bb240e95/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:20.562810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4497039-f3dd-467c-97b2-890e35f6c670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=881128a82b8d4fb1a0b500801ee5397e, user_id=bb7efb83d5e645249213bb11bb240e95, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:20.565575 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4497039-f3dd-467c-97b2-890e35f6c670 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:20.586933 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1002/2018] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:20 2022] PUT /identity/v3/projects/881128a82b8d4fb1a0b500801ee5397e/users/bb7efb83d5e645249213bb11bb240e95/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:13:20.590939 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa517c-4152-469a-949e-8cdef8606d40 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:20.624427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa517c-4152-469a-949e-8cdef8606d40 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:20.726669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05205387-78a3-44d6-af1d-27d692c96974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:20.727692 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05205387-78a3-44d6-af1d-27d692c96974 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:20.728446 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05205387-78a3-44d6-af1d-27d692c96974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:20.728896 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05205387-78a3-44d6-af1d-27d692c96974 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:20.729766 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05205387-78a3-44d6-af1d-27d692c96974 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:20.732095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05205387-78a3-44d6-af1d-27d692c96974 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:20.762362 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1017/2019] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:20.766883 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4497039-f3dd-467c-97b2-890e35f6c670 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:20.793662 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4497039-f3dd-467c-97b2-890e35f6c670 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:20.798168 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3cc2e6-29fa-4fbf-8287-16847b27a904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:20.800394 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cc2e6-29fa-4fbf-8287-16847b27a904 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:20.800794 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cc2e6-29fa-4fbf-8287-16847b27a904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:20.801474 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cc2e6-29fa-4fbf-8287-16847b27a904 admin admin] PATH_INFO: `/v3/projects/881128a82b8d4fb1a0b500801ee5397e/users/bb7efb83d5e645249213bb11bb240e95/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:20.804073 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3cc2e6-29fa-4fbf-8287-16847b27a904 admin admin] RBAC: Authorizing `identity:create_grant(project_id=881128a82b8d4fb1a0b500801ee5397e, user_id=bb7efb83d5e645249213bb11bb240e95, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:20.807005 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3cc2e6-29fa-4fbf-8287-16847b27a904 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:20.822919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1003/2020] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:20 2022] PUT /identity/v3/projects/881128a82b8d4fb1a0b500801ee5397e/users/bb7efb83d5e645249213bb11bb240e95/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:20.835040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e060756-bcc0-4c6e-b498-1c0f75089c8b None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:20.835741 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e060756-bcc0-4c6e-b498-1c0f75089c8b None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:20.836313 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e060756-bcc0-4c6e-b498-1c0f75089c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:20.898951 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-7e060756-bcc0-4c6e-b498-1c0f75089c8b None None] MFA Rules not processed for user `bb7efb83d5e645249213bb11bb240e95`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:20.900805 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-7e060756-bcc0-4c6e-b498-1c0f75089c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:21.084178 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1018/2021] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:20 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:21.114175 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3cc2e6-29fa-4fbf-8287-16847b27a904 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:21.116442 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-0e3cc2e6-29fa-4fbf-8287-16847b27a904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:21.201091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3cc2e6-29fa-4fbf-8287-16847b27a904 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:21.206241 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517d781-7deb-43f5-9264-c552a5d5ee4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:21.207350 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517d781-7deb-43f5-9264-c552a5d5ee4f service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:21.207734 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517d781-7deb-43f5-9264-c552a5d5ee4f service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:21.208167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517d781-7deb-43f5-9264-c552a5d5ee4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:21.213341 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-0517d781-7deb-43f5-9264-c552a5d5ee4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:21.258893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e060756-bcc0-4c6e-b498-1c0f75089c8b None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:21.261292 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-7e060756-bcc0-4c6e-b498-1c0f75089c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:21.275202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:21.370146 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0517d781-7deb-43f5-9264-c552a5d5ee4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:21.375710 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:21.376623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0517d781-7deb-43f5-9264-c552a5d5ee4f service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:21.407887 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1004/2022] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:21.428693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e060756-bcc0-4c6e-b498-1c0f75089c8b None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:21.433549 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7527304d-698c-425f-ab6e-02919ce8dd6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:21.434197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527304d-698c-425f-ab6e-02919ce8dd6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:21.435113 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527304d-698c-425f-ab6e-02919ce8dd6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:21.435113 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527304d-698c-425f-ab6e-02919ce8dd6d admin admin] PATH_INFO: `/v3/projects/b401a6807f1645339ef62198d9a55ab7` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:21.436000 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7527304d-698c-425f-ab6e-02919ce8dd6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b401a6807f1645339ef62198d9a55ab7)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:21.437950 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7527304d-698c-425f-ab6e-02919ce8dd6d admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:21.501515 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-7527304d-698c-425f-ab6e-02919ce8dd6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b401a6807f1645339ef62198d9a55ab7', 'request_id': 'req-7527304d-698c-425f-ab6e-02919ce8dd6d'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:21.508968 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-7527304d-698c-425f-ab6e-02919ce8dd6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b401a6807f1645339ef62198d9a55ab7', 'request_id': 'req-7527304d-698c-425f-ab6e-02919ce8dd6d'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:21.516605 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-7527304d-698c-425f-ab6e-02919ce8dd6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b401a6807f1645339ef62198d9a55ab7', 'request_id': 'req-7527304d-698c-425f-ab6e-02919ce8dd6d'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:21.537747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-7527304d-698c-425f-ab6e-02919ce8dd6d admin admin] The token cache is being invalidate because project b401a6807f1645339ef62198d9a55ab7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:21.538205 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-7527304d-698c-425f-ab6e-02919ce8dd6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:21.539662 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-7527304d-698c-425f-ab6e-02919ce8dd6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:21.543363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1019/2023] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:21 2022] DELETE /identity/v3/projects/b401a6807f1645339ef62198d9a55ab7 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:21.547316 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517d781-7deb-43f5-9264-c552a5d5ee4f service cinder] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:21.562298 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517d781-7deb-43f5-9264-c552a5d5ee4f service cinder] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:21.649121 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20236a1d-54f0-4973-88bf-266d758d29f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:21.651410 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20236a1d-54f0-4973-88bf-266d758d29f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:21.652015 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20236a1d-54f0-4973-88bf-266d758d29f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:21.652535 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20236a1d-54f0-4973-88bf-266d758d29f9 admin admin] PATH_INFO: `/v3/users/1255b48fe9c0435e8de043262fe72370` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:21.653754 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20236a1d-54f0-4973-88bf-266d758d29f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1255b48fe9c0435e8de043262fe72370)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:21.657313 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20236a1d-54f0-4973-88bf-266d758d29f9 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:21.738616 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-20236a1d-54f0-4973-88bf-266d758d29f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1255b48fe9c0435e8de043262fe72370'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:21.749559 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-20236a1d-54f0-4973-88bf-266d758d29f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1255b48fe9c0435e8de043262fe72370'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:21.774469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-20236a1d-54f0-4973-88bf-266d758d29f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1255b48fe9c0435e8de043262fe72370'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:21.775613 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-20236a1d-54f0-4973-88bf-266d758d29f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1255b48fe9c0435e8de043262fe72370'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:21.785709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-20236a1d-54f0-4973-88bf-266d758d29f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1255b48fe9c0435e8de043262fe72370'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:21.796030 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1005/2024] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:21 2022] DELETE /identity/v3/users/1255b48fe9c0435e8de043262fe72370 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:22.701120 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7527304d-698c-425f-ab6e-02919ce8dd6d admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:22.702509 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-7527304d-698c-425f-ab6e-02919ce8dd6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:22.730490 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7527304d-698c-425f-ab6e-02919ce8dd6d admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:22.800530 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd9e4e2-c53e-45d1-8773-13718a04c133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:22.801225 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9e4e2-c53e-45d1-8773-13718a04c133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:22.801523 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9e4e2-c53e-45d1-8773-13718a04c133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:22.801790 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9e4e2-c53e-45d1-8773-13718a04c133 admin admin] PATH_INFO: `/v3/projects/e651265c6b4a4383aec48d23fc0b7776` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:22.802953 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd9e4e2-c53e-45d1-8773-13718a04c133 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e651265c6b4a4383aec48d23fc0b7776)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:22.804928 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd9e4e2-c53e-45d1-8773-13718a04c133 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:22.860831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-7bd9e4e2-c53e-45d1-8773-13718a04c133 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e651265c6b4a4383aec48d23fc0b7776', 'request_id': 'req-7bd9e4e2-c53e-45d1-8773-13718a04c133'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:22.869087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-7bd9e4e2-c53e-45d1-8773-13718a04c133 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e651265c6b4a4383aec48d23fc0b7776', 'request_id': 'req-7bd9e4e2-c53e-45d1-8773-13718a04c133'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:22.876782 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-7bd9e4e2-c53e-45d1-8773-13718a04c133 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e651265c6b4a4383aec48d23fc0b7776', 'request_id': 'req-7bd9e4e2-c53e-45d1-8773-13718a04c133'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:22.890590 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:22.898165 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-7bd9e4e2-c53e-45d1-8773-13718a04c133 admin admin] The token cache is being invalidate because project e651265c6b4a4383aec48d23fc0b7776 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:22.899036 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-7bd9e4e2-c53e-45d1-8773-13718a04c133 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:22.901240 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-7bd9e4e2-c53e-45d1-8773-13718a04c133 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:22.904061 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1020/2025] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:22 2022] DELETE /identity/v3/projects/e651265c6b4a4383aec48d23fc0b7776 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:22.910235 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5042e-1cbc-446d-be8a-c7ccf745a7dd None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:22.911410 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5042e-1cbc-446d-be8a-c7ccf745a7dd None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:22.911837 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5042e-1cbc-446d-be8a-c7ccf745a7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:22.939816 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-eba5042e-1cbc-446d-be8a-c7ccf745a7dd None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:22.949486 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-eba5042e-1cbc-446d-be8a-c7ccf745a7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:23.007803 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:23.031878 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1006/2026] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:23.040373 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd81188-9307-4890-98e1-ddea185c397f None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:23.040649 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd81188-9307-4890-98e1-ddea185c397f None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:23.041322 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd81188-9307-4890-98e1-ddea185c397f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:23.062636 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-7fd81188-9307-4890-98e1-ddea185c397f None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:23.064192 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-7fd81188-9307-4890-98e1-ddea185c397f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:23.077216 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1021/2027] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:13:23.077630 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5042e-1cbc-446d-be8a-c7ccf745a7dd None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:23.089683 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5042e-1cbc-446d-be8a-c7ccf745a7dd None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:23.103220 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b02aab-1f5e-487b-b2f3-4c43c3e6b4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:23.103949 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b02aab-1f5e-487b-b2f3-4c43c3e6b4ce admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:23.104215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b02aab-1f5e-487b-b2f3-4c43c3e6b4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:23.105002 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b02aab-1f5e-487b-b2f3-4c43c3e6b4ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:23.105157 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b02aab-1f5e-487b-b2f3-4c43c3e6b4ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:13:23.105439 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b02aab-1f5e-487b-b2f3-4c43c3e6b4ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:23.107072 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b02aab-1f5e-487b-b2f3-4c43c3e6b4ce admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:23.128972 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1007/2028] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:13:23.130299 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd81188-9307-4890-98e1-ddea185c397f None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:23.143873 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd81188-9307-4890-98e1-ddea185c397f None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:23.152542 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc37535-404c-4477-8beb-8628dd6ae12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:23.154350 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc37535-404c-4477-8beb-8628dd6ae12f admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:23.156108 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc37535-404c-4477-8beb-8628dd6ae12f admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:23.156633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc37535-404c-4477-8beb-8628dd6ae12f admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:23.159237 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc37535-404c-4477-8beb-8628dd6ae12f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:23.167621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc37535-404c-4477-8beb-8628dd6ae12f admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:23.219959 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1022/2029] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:13:23 2022] POST /identity/v3/projects => generated 399 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:23.220855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b02aab-1f5e-487b-b2f3-4c43c3e6b4ce admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:23.239138 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b02aab-1f5e-487b-b2f3-4c43c3e6b4ce admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:23.336786 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ae860-8df5-47ae-828d-c6985a76ed8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:23.337439 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ae860-8df5-47ae-828d-c6985a76ed8a admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:23.337707 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ae860-8df5-47ae-828d-c6985a76ed8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:23.337967 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ae860-8df5-47ae-828d-c6985a76ed8a admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:23.338593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ae860-8df5-47ae-828d-c6985a76ed8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:23.340509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ae860-8df5-47ae-828d-c6985a76ed8a admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:23.369684 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1008/2030] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:13:23 2022] POST /identity/v3/users => generated 300 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:23.371919 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc37535-404c-4477-8beb-8628dd6ae12f admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:23.383824 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc37535-404c-4477-8beb-8628dd6ae12f admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:23.388942 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228a9c1-d00a-40d6-980c-f89385fde35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:23.389630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228a9c1-d00a-40d6-980c-f89385fde35c admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:23.389908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228a9c1-d00a-40d6-980c-f89385fde35c admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:23.390176 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228a9c1-d00a-40d6-980c-f89385fde35c admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:23.390830 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8228a9c1-d00a-40d6-980c-f89385fde35c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:23.392844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8228a9c1-d00a-40d6-980c-f89385fde35c admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:23.406152 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1023/2031] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:23.409402 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ae860-8df5-47ae-828d-c6985a76ed8a admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:23.424101 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ae860-8df5-47ae-828d-c6985a76ed8a admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:23.429201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6609e8a-0bbf-40ae-9290-4abb76a40bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:23.429932 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6609e8a-0bbf-40ae-9290-4abb76a40bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:23.430276 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6609e8a-0bbf-40ae-9290-4abb76a40bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:23.430664 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6609e8a-0bbf-40ae-9290-4abb76a40bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:23.431397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6609e8a-0bbf-40ae-9290-4abb76a40bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:23.433247 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6609e8a-0bbf-40ae-9290-4abb76a40bc8 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:23.445779 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1009/2032] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:23.450543 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228a9c1-d00a-40d6-980c-f89385fde35c admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:23.462029 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228a9c1-d00a-40d6-980c-f89385fde35c admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:23.466877 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fea346-178a-4ffa-974d-5beff0f603ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:23.467570 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fea346-178a-4ffa-974d-5beff0f603ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:23.467847 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fea346-178a-4ffa-974d-5beff0f603ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:23.468115 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fea346-178a-4ffa-974d-5beff0f603ff admin admin] PATH_INFO: `/v3/projects/f5630402ddbb47bdac6749d48d687886/users/8eceba5a46194b9f9d34f0f60573cbe1/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:23.484117 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fea346-178a-4ffa-974d-5beff0f603ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5630402ddbb47bdac6749d48d687886, user_id=8eceba5a46194b9f9d34f0f60573cbe1, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:23.486828 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fea346-178a-4ffa-974d-5beff0f603ff admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:23.503122 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1024/2033] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:23 2022] PUT /identity/v3/projects/f5630402ddbb47bdac6749d48d687886/users/8eceba5a46194b9f9d34f0f60573cbe1/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:23.513796 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c77f93-3f46-49dd-977a-f6f3cefe739e None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:23.514075 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c77f93-3f46-49dd-977a-f6f3cefe739e None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:23.514388 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c77f93-3f46-49dd-977a-f6f3cefe739e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:23.544385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-c4c77f93-3f46-49dd-977a-f6f3cefe739e None None] MFA Rules not processed for user `8eceba5a46194b9f9d34f0f60573cbe1`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:23.545635 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c4c77f93-3f46-49dd-977a-f6f3cefe739e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:23.614053 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1010/2034] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:23 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:23.618891 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fea346-178a-4ffa-974d-5beff0f603ff admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:23.689050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fea346-178a-4ffa-974d-5beff0f603ff admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:23.694507 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f21a4-22d5-4d17-827c-e3c9932306ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:23.695134 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f21a4-22d5-4d17-827c-e3c9932306ee admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:23.695407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f21a4-22d5-4d17-827c-e3c9932306ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:23.695682 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f21a4-22d5-4d17-827c-e3c9932306ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:23.696363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140f21a4-22d5-4d17-827c-e3c9932306ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:23.698280 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140f21a4-22d5-4d17-827c-e3c9932306ee admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:23.736882 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1025/2035] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:13:23 2022] POST /identity/v3/projects => generated 399 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:23.738831 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c77f93-3f46-49dd-977a-f6f3cefe739e None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:23.750329 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c77f93-3f46-49dd-977a-f6f3cefe739e None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:23.792833 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:23.810359 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5e5fe-f1a3-4f98-9407-a3da9e569851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:23.811005 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5e5fe-f1a3-4f98-9407-a3da9e569851 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:23.811271 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5e5fe-f1a3-4f98-9407-a3da9e569851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:23.811533 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5e5fe-f1a3-4f98-9407-a3da9e569851 admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:23.812177 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5e5fe-f1a3-4f98-9407-a3da9e569851 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:23.814204 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5e5fe-f1a3-4f98-9407-a3da9e569851 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:23.837407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1011/2036] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:13:23 2022] POST /identity/v3/users => generated 306 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:23.842699 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f21a4-22d5-4d17-827c-e3c9932306ee admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:23.853294 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f21a4-22d5-4d17-827c-e3c9932306ee admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:23.858040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f1a198-9610-4585-af4f-5f37ba33fc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:23.858643 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1a198-9610-4585-af4f-5f37ba33fc4c admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:23.858917 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1a198-9610-4585-af4f-5f37ba33fc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:23.859186 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1a198-9610-4585-af4f-5f37ba33fc4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:23.859833 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f1a198-9610-4585-af4f-5f37ba33fc4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:23.861476 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f1a198-9610-4585-af4f-5f37ba33fc4c admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:23.874761 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1026/2037] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:13:23.879537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5e5fe-f1a3-4f98-9407-a3da9e569851 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:23.889976 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5e5fe-f1a3-4f98-9407-a3da9e569851 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:23.894660 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75ada0-492e-4196-ae28-7a902b57e240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:23.895352 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75ada0-492e-4196-ae28-7a902b57e240 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:23.895637 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75ada0-492e-4196-ae28-7a902b57e240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:23.895910 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75ada0-492e-4196-ae28-7a902b57e240 admin admin] PATH_INFO: `/v3/projects/bd857fb99f70455c9d44ee6cbae0496f/users/3fcb73e264ea40968535fd8dd7e57a5e/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:23.912882 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a75ada0-492e-4196-ae28-7a902b57e240 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd857fb99f70455c9d44ee6cbae0496f, user_id=3fcb73e264ea40968535fd8dd7e57a5e, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:23.915602 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a75ada0-492e-4196-ae28-7a902b57e240 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:23.930910 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1012/2038] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:23 2022] PUT /identity/v3/projects/bd857fb99f70455c9d44ee6cbae0496f/users/3fcb73e264ea40968535fd8dd7e57a5e/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:23.935107 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f1a198-9610-4585-af4f-5f37ba33fc4c admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:23.946265 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f1a198-9610-4585-af4f-5f37ba33fc4c admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:24.022764 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67545c7e-6e15-43d1-abff-ecbf5ba9a00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:24.023383 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67545c7e-6e15-43d1-abff-ecbf5ba9a00c admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:24.023656 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67545c7e-6e15-43d1-abff-ecbf5ba9a00c admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:24.023923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67545c7e-6e15-43d1-abff-ecbf5ba9a00c admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:24.024566 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67545c7e-6e15-43d1-abff-ecbf5ba9a00c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:24.026205 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67545c7e-6e15-43d1-abff-ecbf5ba9a00c admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:24.038070 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1027/2039] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:24.042547 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75ada0-492e-4196-ae28-7a902b57e240 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:24.053699 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75ada0-492e-4196-ae28-7a902b57e240 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:24.058839 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6986b4-a8c3-4044-a9bb-4b796746d7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:24.059504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6986b4-a8c3-4044-a9bb-4b796746d7f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:24.059778 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6986b4-a8c3-4044-a9bb-4b796746d7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:24.060050 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6986b4-a8c3-4044-a9bb-4b796746d7f7 admin admin] PATH_INFO: `/v3/projects/bd857fb99f70455c9d44ee6cbae0496f/users/3fcb73e264ea40968535fd8dd7e57a5e/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:24.062037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6986b4-a8c3-4044-a9bb-4b796746d7f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd857fb99f70455c9d44ee6cbae0496f, user_id=3fcb73e264ea40968535fd8dd7e57a5e, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:24.064804 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6986b4-a8c3-4044-a9bb-4b796746d7f7 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:24.077994 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1013/2040] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:24 2022] PUT /identity/v3/projects/bd857fb99f70455c9d44ee6cbae0496f/users/3fcb73e264ea40968535fd8dd7e57a5e/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:24.089441 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c044ab-9676-4700-bdfa-7a17eee3cc8a None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:24.089839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c044ab-9676-4700-bdfa-7a17eee3cc8a None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:24.090251 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c044ab-9676-4700-bdfa-7a17eee3cc8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:24.099146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:24.124440 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-f2c044ab-9676-4700-bdfa-7a17eee3cc8a None None] MFA Rules not processed for user `3fcb73e264ea40968535fd8dd7e57a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:24.126008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-f2c044ab-9676-4700-bdfa-7a17eee3cc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:24.200724 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1028/2041] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:24 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:24.212837 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6986b4-a8c3-4044-a9bb-4b796746d7f7 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:24.214752 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-9a6986b4-a8c3-4044-a9bb-4b796746d7f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:24.281878 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6986b4-a8c3-4044-a9bb-4b796746d7f7 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:24.285668 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f307678-cff0-4552-8847-3230d23ae1cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:24.286282 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f307678-cff0-4552-8847-3230d23ae1cd service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:24.286550 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f307678-cff0-4552-8847-3230d23ae1cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:24.286814 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f307678-cff0-4552-8847-3230d23ae1cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:24.296578 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f307678-cff0-4552-8847-3230d23ae1cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:24.298055 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f307678-cff0-4552-8847-3230d23ae1cd service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:24.309384 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1014/2042] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:24 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:24.492527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c044ab-9676-4700-bdfa-7a17eee3cc8a None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:24.502836 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c044ab-9676-4700-bdfa-7a17eee3cc8a None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:24.507418 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bdff92-ee2a-4537-b111-39ef9bede24d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:24.508154 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bdff92-ee2a-4537-b111-39ef9bede24d service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:24.508490 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bdff92-ee2a-4537-b111-39ef9bede24d service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:24.508823 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bdff92-ee2a-4537-b111-39ef9bede24d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:24.586783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bdff92-ee2a-4537-b111-39ef9bede24d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:24.588243 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bdff92-ee2a-4537-b111-39ef9bede24d service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:24.598363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1029/2043] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:24 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:13:25.261316 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f307678-cff0-4552-8847-3230d23ae1cd service cinder] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:25.270752 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f307678-cff0-4552-8847-3230d23ae1cd service cinder] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:25.274778 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a2019-b870-49ea-bd44-6715c5232382 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:25.275500 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a2019-b870-49ea-bd44-6715c5232382 service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:25.275775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a2019-b870-49ea-bd44-6715c5232382 service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:25.276039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a2019-b870-49ea-bd44-6715c5232382 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:25.278047 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-146a2019-b870-49ea-bd44-6715c5232382 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:25.350941 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146a2019-b870-49ea-bd44-6715c5232382 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:25.352596 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146a2019-b870-49ea-bd44-6715c5232382 service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:25.362919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1015/2044] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:26.033540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bdff92-ee2a-4537-b111-39ef9bede24d service cinder] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:26.044200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-a0bdff92-ee2a-4537-b111-39ef9bede24d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:26.056921 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bdff92-ee2a-4537-b111-39ef9bede24d service cinder] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:26.061157 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729d207-ff6d-417f-b4c8-71aa2a3462e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:26.061785 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729d207-ff6d-417f-b4c8-71aa2a3462e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:26.062057 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729d207-ff6d-417f-b4c8-71aa2a3462e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:26.062324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729d207-ff6d-417f-b4c8-71aa2a3462e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:26.064137 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-4729d207-ff6d-417f-b4c8-71aa2a3462e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:26.132208 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4729d207-ff6d-417f-b4c8-71aa2a3462e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:26.133806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4729d207-ff6d-417f-b4c8-71aa2a3462e5 service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:26.144810 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1030/2045] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:26 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:30.319449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a2019-b870-49ea-bd44-6715c5232382 service cinder] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:30.320972 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-146a2019-b870-49ea-bd44-6715c5232382 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:30.363533 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:30.392118 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a2019-b870-49ea-bd44-6715c5232382 service cinder] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:30.397637 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07a8ce-c4d7-4454-a3eb-33e09261fc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:30.398284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07a8ce-c4d7-4454-a3eb-33e09261fc7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:30.399325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07a8ce-c4d7-4454-a3eb-33e09261fc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:30.399325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07a8ce-c4d7-4454-a3eb-33e09261fc7f admin admin] PATH_INFO: `/v3/users/5649a36703744e28bfbbdc18331c16fa` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:30.400086 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d07a8ce-c4d7-4454-a3eb-33e09261fc7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5649a36703744e28bfbbdc18331c16fa)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:30.402095 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d07a8ce-c4d7-4454-a3eb-33e09261fc7f admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:30.459858 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3d07a8ce-c4d7-4454-a3eb-33e09261fc7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5649a36703744e28bfbbdc18331c16fa'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:30.470332 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3d07a8ce-c4d7-4454-a3eb-33e09261fc7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5649a36703744e28bfbbdc18331c16fa'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:30.486243 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3d07a8ce-c4d7-4454-a3eb-33e09261fc7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5649a36703744e28bfbbdc18331c16fa'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:30.486983 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3d07a8ce-c4d7-4454-a3eb-33e09261fc7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5649a36703744e28bfbbdc18331c16fa'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:30.494627 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3d07a8ce-c4d7-4454-a3eb-33e09261fc7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5649a36703744e28bfbbdc18331c16fa'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:30.501180 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1016/2046] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:30 2022] DELETE /identity/v3/users/5649a36703744e28bfbbdc18331c16fa => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:30.505596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9d05d-532e-40fc-b75b-2c18a5b88871 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:30.506109 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9d05d-532e-40fc-b75b-2c18a5b88871 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:30.506217 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9d05d-532e-40fc-b75b-2c18a5b88871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:30.525269 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-fcc9d05d-532e-40fc-b75b-2c18a5b88871 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:30.526494 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-fcc9d05d-532e-40fc-b75b-2c18a5b88871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:30.543891 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:30.590954 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1031/2047] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:30.601899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07a8ce-c4d7-4454-a3eb-33e09261fc7f admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:30.607012 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-3d07a8ce-c4d7-4454-a3eb-33e09261fc7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:30.667328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07a8ce-c4d7-4454-a3eb-33e09261fc7f admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:30.671905 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cea2031-28a7-4c45-a7fe-8f0440482988 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:30.672566 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea2031-28a7-4c45-a7fe-8f0440482988 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:30.672829 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea2031-28a7-4c45-a7fe-8f0440482988 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:30.673102 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea2031-28a7-4c45-a7fe-8f0440482988 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:30.694532 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cea2031-28a7-4c45-a7fe-8f0440482988 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:30.705899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cea2031-28a7-4c45-a7fe-8f0440482988 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:30.725253 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1017/2048] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:30.938748 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9d05d-532e-40fc-b75b-2c18a5b88871 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:30.940556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-fcc9d05d-532e-40fc-b75b-2c18a5b88871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:30.961169 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9d05d-532e-40fc-b75b-2c18a5b88871 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:30.966285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0736b2ea-3e39-458b-887e-94394878e614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:30.966943 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736b2ea-3e39-458b-887e-94394878e614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:30.967212 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736b2ea-3e39-458b-887e-94394878e614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:30.967475 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736b2ea-3e39-458b-887e-94394878e614 admin admin] PATH_INFO: `/v3/users/8eceba5a46194b9f9d34f0f60573cbe1` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:30.968644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0736b2ea-3e39-458b-887e-94394878e614 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eceba5a46194b9f9d34f0f60573cbe1)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:30.970640 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0736b2ea-3e39-458b-887e-94394878e614 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:31.018704 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-0736b2ea-3e39-458b-887e-94394878e614 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eceba5a46194b9f9d34f0f60573cbe1'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:31.027174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-0736b2ea-3e39-458b-887e-94394878e614 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eceba5a46194b9f9d34f0f60573cbe1'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:31.041790 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-0736b2ea-3e39-458b-887e-94394878e614 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eceba5a46194b9f9d34f0f60573cbe1'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:31.042629 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-0736b2ea-3e39-458b-887e-94394878e614 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eceba5a46194b9f9d34f0f60573cbe1'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:31.049837 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-0736b2ea-3e39-458b-887e-94394878e614 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eceba5a46194b9f9d34f0f60573cbe1'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:31.056699 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1032/2049] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:30 2022] DELETE /identity/v3/users/8eceba5a46194b9f9d34f0f60573cbe1 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:31.061808 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6a4ed-2938-48ac-847f-3ac7126439f4 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:31.064396 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6a4ed-2938-48ac-847f-3ac7126439f4 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:31.064396 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6a4ed-2938-48ac-847f-3ac7126439f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:31.085611 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-53c6a4ed-2938-48ac-847f-3ac7126439f4 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:31.086927 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-53c6a4ed-2938-48ac-847f-3ac7126439f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:31.091792 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0736b2ea-3e39-458b-887e-94394878e614 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:31.093812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-0736b2ea-3e39-458b-887e-94394878e614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:31.152399 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:31.177260 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0736b2ea-3e39-458b-887e-94394878e614 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:31.179809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1018/2050] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:31.182614 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f6471f-6861-4de2-b75c-a07e7ab83de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:31.183343 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6471f-6861-4de2-b75c-a07e7ab83de3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:31.183622 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6471f-6861-4de2-b75c-a07e7ab83de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:31.183893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6471f-6861-4de2-b75c-a07e7ab83de3 admin admin] PATH_INFO: `/v3/projects/005a5a842d0f4f4ba39af249fa0f9121` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:31.185012 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f6471f-6861-4de2-b75c-a07e7ab83de3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=005a5a842d0f4f4ba39af249fa0f9121)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:31.186947 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f6471f-6861-4de2-b75c-a07e7ab83de3 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:31.193600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6a4ed-2938-48ac-847f-3ac7126439f4 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:31.195018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-53c6a4ed-2938-48ac-847f-3ac7126439f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:31.227363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:31.237736 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-24f6471f-6861-4de2-b75c-a07e7ab83de3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '005a5a842d0f4f4ba39af249fa0f9121', 'request_id': 'req-24f6471f-6861-4de2-b75c-a07e7ab83de3'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:31.244752 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-24f6471f-6861-4de2-b75c-a07e7ab83de3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '005a5a842d0f4f4ba39af249fa0f9121', 'request_id': 'req-24f6471f-6861-4de2-b75c-a07e7ab83de3'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:31.251364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-24f6471f-6861-4de2-b75c-a07e7ab83de3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '005a5a842d0f4f4ba39af249fa0f9121', 'request_id': 'req-24f6471f-6861-4de2-b75c-a07e7ab83de3'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:31.262903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6a4ed-2938-48ac-847f-3ac7126439f4 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:31.266496 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-24f6471f-6861-4de2-b75c-a07e7ab83de3 admin admin] The token cache is being invalidate because project 005a5a842d0f4f4ba39af249fa0f9121 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:31.266836 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-24f6471f-6861-4de2-b75c-a07e7ab83de3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:31.267462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-24f6471f-6861-4de2-b75c-a07e7ab83de3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:31.269071 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1033/2051] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:31 2022] DELETE /identity/v3/projects/005a5a842d0f4f4ba39af249fa0f9121 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:31.278260 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c5249-c532-44f5-a959-c2159bd56125 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:31.278657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c5249-c532-44f5-a959-c2159bd56125 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:31.279003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c5249-c532-44f5-a959-c2159bd56125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:31.298724 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-704c5249-c532-44f5-a959-c2159bd56125 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:31.299905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-704c5249-c532-44f5-a959-c2159bd56125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:31.312353 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc26c18-26a8-4043-aa88-2c3dfb31ded2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:31.313236 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc26c18-26a8-4043-aa88-2c3dfb31ded2 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:31.313524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc26c18-26a8-4043-aa88-2c3dfb31ded2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:31.313793 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc26c18-26a8-4043-aa88-2c3dfb31ded2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:31.315549 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-3fc26c18-26a8-4043-aa88-2c3dfb31ded2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:31.363881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1034/2052] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:31.371632 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1e137-f72f-4e1d-bf5c-af5b4fc61deb None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:31.371918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1e137-f72f-4e1d-bf5c-af5b4fc61deb None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:31.372190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1e137-f72f-4e1d-bf5c-af5b4fc61deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:31.381403 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc26c18-26a8-4043-aa88-2c3dfb31ded2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:31.382902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc26c18-26a8-4043-aa88-2c3dfb31ded2 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:31.392441 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-5cd1e137-f72f-4e1d-bf5c-af5b4fc61deb None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:31.392615 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1019/2053] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:13:31.393674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-5cd1e137-f72f-4e1d-bf5c-af5b4fc61deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:31.402641 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1035/2054] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:13:31.407057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc26c18-26a8-4043-aa88-2c3dfb31ded2 service neutron] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:31.417004 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc26c18-26a8-4043-aa88-2c3dfb31ded2 service neutron] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:31.422680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a17d62-b34c-4e6a-a1b8-e9ad24d2f6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:31.423331 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a17d62-b34c-4e6a-a1b8-e9ad24d2f6c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:31.423609 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a17d62-b34c-4e6a-a1b8-e9ad24d2f6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:31.424033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a17d62-b34c-4e6a-a1b8-e9ad24d2f6c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:31.424687 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a17d62-b34c-4e6a-a1b8-e9ad24d2f6c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:13:31.424995 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a17d62-b34c-4e6a-a1b8-e9ad24d2f6c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:31.426719 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a17d62-b34c-4e6a-a1b8-e9ad24d2f6c5 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:31.442711 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1020/2055] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:13:31.444057 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd1e137-f72f-4e1d-bf5c-af5b4fc61deb None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:31.457727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd1e137-f72f-4e1d-bf5c-af5b4fc61deb None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:31.464430 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf0813-cf2e-4794-bdca-eb183fb9d05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:31.465572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf0813-cf2e-4794-bdca-eb183fb9d05d admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:31.466078 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf0813-cf2e-4794-bdca-eb183fb9d05d admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:31.466538 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf0813-cf2e-4794-bdca-eb183fb9d05d admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:31.467240 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bf0813-cf2e-4794-bdca-eb183fb9d05d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:31.469382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bf0813-cf2e-4794-bdca-eb183fb9d05d admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:31.502735 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1036/2056] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:13:31 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:31.507979 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a17d62-b34c-4e6a-a1b8-e9ad24d2f6c5 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:31.518629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a17d62-b34c-4e6a-a1b8-e9ad24d2f6c5 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:31.595825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:31.602753 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0700d3-b096-499d-af13-3a7c4c751964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:31.603632 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0700d3-b096-499d-af13-3a7c4c751964 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:31.604160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0700d3-b096-499d-af13-3a7c4c751964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:31.604518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0700d3-b096-499d-af13-3a7c4c751964 admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:31.605504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0700d3-b096-499d-af13-3a7c4c751964 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:31.607776 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0700d3-b096-499d-af13-3a7c4c751964 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:31.635850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1021/2057] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:13:31 2022] POST /identity/v3/users => generated 293 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:31.640329 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf0813-cf2e-4794-bdca-eb183fb9d05d admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:31.653554 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf0813-cf2e-4794-bdca-eb183fb9d05d admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:31.660680 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23943f-28da-4370-b2df-2acac67790d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:31.661555 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23943f-28da-4370-b2df-2acac67790d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:31.661971 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23943f-28da-4370-b2df-2acac67790d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:31.662450 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23943f-28da-4370-b2df-2acac67790d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:31.663294 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23943f-28da-4370-b2df-2acac67790d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:31.665410 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23943f-28da-4370-b2df-2acac67790d2 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:31.678705 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1037/2058] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:31.683899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0700d3-b096-499d-af13-3a7c4c751964 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:31.696629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0700d3-b096-499d-af13-3a7c4c751964 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:31.710411 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23943f-28da-4370-b2df-2acac67790d2 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:31.711123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969436f9-8086-4637-8dd9-2ab27bc21fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:31.711768 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969436f9-8086-4637-8dd9-2ab27bc21fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:31.712118 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969436f9-8086-4637-8dd9-2ab27bc21fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:31.712349 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969436f9-8086-4637-8dd9-2ab27bc21fe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:31.712612 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-6c23943f-28da-4370-b2df-2acac67790d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:31.713437 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969436f9-8086-4637-8dd9-2ab27bc21fe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:31.725457 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969436f9-8086-4637-8dd9-2ab27bc21fe6 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:31.727101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23943f-28da-4370-b2df-2acac67790d2 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:31.736005 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b433ad-b05d-46c4-91fd-ca2be33eaa4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:31.736793 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b433ad-b05d-46c4-91fd-ca2be33eaa4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:31.737200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b433ad-b05d-46c4-91fd-ca2be33eaa4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:31.737501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b433ad-b05d-46c4-91fd-ca2be33eaa4d admin admin] PATH_INFO: `/v3/projects/f5630402ddbb47bdac6749d48d687886` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:31.738820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b433ad-b05d-46c4-91fd-ca2be33eaa4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5630402ddbb47bdac6749d48d687886)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:31.740932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b433ad-b05d-46c4-91fd-ca2be33eaa4d admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:31.742256 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1022/2059] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:31.743392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969436f9-8086-4637-8dd9-2ab27bc21fe6 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:31.753669 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969436f9-8086-4637-8dd9-2ab27bc21fe6 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:31.758567 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ebfc3-1a77-4c9d-bb35-019942137154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:31.759223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ebfc3-1a77-4c9d-bb35-019942137154 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:31.759495 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ebfc3-1a77-4c9d-bb35-019942137154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:31.759755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ebfc3-1a77-4c9d-bb35-019942137154 admin admin] PATH_INFO: `/v3/projects/d608a9b9311d47bc8933913d4f88a26e/users/dc5042d69e604dc999fc9100897e6b82/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:31.776097 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ebfc3-1a77-4c9d-bb35-019942137154 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d608a9b9311d47bc8933913d4f88a26e, user_id=dc5042d69e604dc999fc9100897e6b82, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:31.778958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ebfc3-1a77-4c9d-bb35-019942137154 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:31.798692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1023/2060] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:31 2022] PUT /identity/v3/projects/d608a9b9311d47bc8933913d4f88a26e/users/dc5042d69e604dc999fc9100897e6b82/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:31.804205 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-80b433ad-b05d-46c4-91fd-ca2be33eaa4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5630402ddbb47bdac6749d48d687886', 'request_id': 'req-80b433ad-b05d-46c4-91fd-ca2be33eaa4d'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:31.806652 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e61f8-9bad-4f4d-a8d2-1b2b434453bc None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:31.806820 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e61f8-9bad-4f4d-a8d2-1b2b434453bc None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:31.807084 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e61f8-9bad-4f4d-a8d2-1b2b434453bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:31.810832 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-80b433ad-b05d-46c4-91fd-ca2be33eaa4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5630402ddbb47bdac6749d48d687886', 'request_id': 'req-80b433ad-b05d-46c4-91fd-ca2be33eaa4d'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:31.822696 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-80b433ad-b05d-46c4-91fd-ca2be33eaa4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5630402ddbb47bdac6749d48d687886', 'request_id': 'req-80b433ad-b05d-46c4-91fd-ca2be33eaa4d'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:31.839490 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-80b433ad-b05d-46c4-91fd-ca2be33eaa4d admin admin] The token cache is being invalidate because project f5630402ddbb47bdac6749d48d687886 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:31.839781 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-80b433ad-b05d-46c4-91fd-ca2be33eaa4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:31.840439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-80b433ad-b05d-46c4-91fd-ca2be33eaa4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:31.845253 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1038/2061] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:31 2022] DELETE /identity/v3/projects/f5630402ddbb47bdac6749d48d687886 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:31.846664 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b433ad-b05d-46c4-91fd-ca2be33eaa4d admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:31.848566 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-80b433ad-b05d-46c4-91fd-ca2be33eaa4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:31.850725 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-b22e61f8-9bad-4f4d-a8d2-1b2b434453bc None None] MFA Rules not processed for user `dc5042d69e604dc999fc9100897e6b82`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:31.852020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b22e61f8-9bad-4f4d-a8d2-1b2b434453bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:31.870274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:31.931389 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b433ad-b05d-46c4-91fd-ca2be33eaa4d admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:31.939726 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1024/2062] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:31.953985 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e61f8-9bad-4f4d-a8d2-1b2b434453bc None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:31.953985 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b22e61f8-9bad-4f4d-a8d2-1b2b434453bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:32.001341 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d596827-05f7-4f23-8963-8ae24b2f2bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:32.001990 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d596827-05f7-4f23-8963-8ae24b2f2bbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:32.002264 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d596827-05f7-4f23-8963-8ae24b2f2bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:32.002532 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d596827-05f7-4f23-8963-8ae24b2f2bbb admin admin] PATH_INFO: `/v3/users/3fcb73e264ea40968535fd8dd7e57a5e` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:32.003684 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d596827-05f7-4f23-8963-8ae24b2f2bbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fcb73e264ea40968535fd8dd7e57a5e)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:32.005672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d596827-05f7-4f23-8963-8ae24b2f2bbb admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:32.012298 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e61f8-9bad-4f4d-a8d2-1b2b434453bc None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:32.016111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99849f13-e4f5-4fe3-a6a2-03a0ec421324 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:32.016733 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99849f13-e4f5-4fe3-a6a2-03a0ec421324 service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:32.017010 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99849f13-e4f5-4fe3-a6a2-03a0ec421324 service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:32.017328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99849f13-e4f5-4fe3-a6a2-03a0ec421324 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:32.026221 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99849f13-e4f5-4fe3-a6a2-03a0ec421324 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:32.027600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99849f13-e4f5-4fe3-a6a2-03a0ec421324 service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:32.037324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1025/2063] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:32.054876 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-1d596827-05f7-4f23-8963-8ae24b2f2bbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fcb73e264ea40968535fd8dd7e57a5e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:32.063237 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-1d596827-05f7-4f23-8963-8ae24b2f2bbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fcb73e264ea40968535fd8dd7e57a5e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:32.076637 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-1d596827-05f7-4f23-8963-8ae24b2f2bbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fcb73e264ea40968535fd8dd7e57a5e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:32.077355 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-1d596827-05f7-4f23-8963-8ae24b2f2bbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fcb73e264ea40968535fd8dd7e57a5e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:32.084562 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-1d596827-05f7-4f23-8963-8ae24b2f2bbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fcb73e264ea40968535fd8dd7e57a5e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:32.090505 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1039/2064] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:31 2022] DELETE /identity/v3/users/3fcb73e264ea40968535fd8dd7e57a5e => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:32.401914 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99849f13-e4f5-4fe3-a6a2-03a0ec421324 service cinder] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:32.417235 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99849f13-e4f5-4fe3-a6a2-03a0ec421324 service cinder] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:32.453011 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:32.477860 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d1741-ab79-4be8-8473-dabbb47589bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:32.478496 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d1741-ab79-4be8-8473-dabbb47589bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:32.478771 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d1741-ab79-4be8-8473-dabbb47589bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:32.479038 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d1741-ab79-4be8-8473-dabbb47589bf admin admin] PATH_INFO: `/v3/projects/bd857fb99f70455c9d44ee6cbae0496f` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:32.480165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d1741-ab79-4be8-8473-dabbb47589bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd857fb99f70455c9d44ee6cbae0496f)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:32.482179 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d1741-ab79-4be8-8473-dabbb47589bf admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:32.531679 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-111d1741-ab79-4be8-8473-dabbb47589bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd857fb99f70455c9d44ee6cbae0496f', 'request_id': 'req-111d1741-ab79-4be8-8473-dabbb47589bf'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:32.540495 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-111d1741-ab79-4be8-8473-dabbb47589bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd857fb99f70455c9d44ee6cbae0496f', 'request_id': 'req-111d1741-ab79-4be8-8473-dabbb47589bf'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:32.547612 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-111d1741-ab79-4be8-8473-dabbb47589bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd857fb99f70455c9d44ee6cbae0496f', 'request_id': 'req-111d1741-ab79-4be8-8473-dabbb47589bf'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:32.559373 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-111d1741-ab79-4be8-8473-dabbb47589bf admin admin] The token cache is being invalidate because project bd857fb99f70455c9d44ee6cbae0496f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:32.559659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-111d1741-ab79-4be8-8473-dabbb47589bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:32.560396 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-111d1741-ab79-4be8-8473-dabbb47589bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:32.562007 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1026/2065] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:32 2022] DELETE /identity/v3/projects/bd857fb99f70455c9d44ee6cbae0496f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:32.567496 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7eeee-f5cb-4e33-916a-74d9a38d9cee None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:32.567775 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7eeee-f5cb-4e33-916a-74d9a38d9cee None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:32.568048 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7eeee-f5cb-4e33-916a-74d9a38d9cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:32.588444 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-03b7eeee-f5cb-4e33-916a-74d9a38d9cee None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:32.589681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-03b7eeee-f5cb-4e33-916a-74d9a38d9cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:32.620244 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:32.653436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1040/2066] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:32.661683 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955e11d-d24d-4195-bf85-c5bc45f8528b None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:32.661954 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955e11d-d24d-4195-bf85-c5bc45f8528b None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:32.662222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955e11d-d24d-4195-bf85-c5bc45f8528b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:32.683473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-2955e11d-d24d-4195-bf85-c5bc45f8528b None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:32.684698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-2955e11d-d24d-4195-bf85-c5bc45f8528b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:32.693451 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1027/2067] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:32.698105 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b7eeee-f5cb-4e33-916a-74d9a38d9cee None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:32.708532 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b7eeee-f5cb-4e33-916a-74d9a38d9cee None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:32.724074 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31f1cd-5df1-4f9a-9164-d7ae39dc10ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:32.725113 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31f1cd-5df1-4f9a-9164-d7ae39dc10ff admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:32.725565 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31f1cd-5df1-4f9a-9164-d7ae39dc10ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:32.726032 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31f1cd-5df1-4f9a-9164-d7ae39dc10ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:32.727050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31f1cd-5df1-4f9a-9164-d7ae39dc10ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:13:32.727563 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31f1cd-5df1-4f9a-9164-d7ae39dc10ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:32.730093 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31f1cd-5df1-4f9a-9164-d7ae39dc10ff admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:32.747003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1041/2068] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:13:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:13:32.748273 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2955e11d-d24d-4195-bf85-c5bc45f8528b None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:32.759266 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2955e11d-d24d-4195-bf85-c5bc45f8528b None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:32.764770 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d96d38-6f45-4ea6-b9f4-c0e54f495030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:32.765411 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d96d38-6f45-4ea6-b9f4-c0e54f495030 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:32.765685 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d96d38-6f45-4ea6-b9f4-c0e54f495030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:32.765950 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d96d38-6f45-4ea6-b9f4-c0e54f495030 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:32.766581 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d96d38-6f45-4ea6-b9f4-c0e54f495030 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:32.768488 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d96d38-6f45-4ea6-b9f4-c0e54f495030 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:32.801975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1028/2069] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:13:32 2022] POST /identity/v3/projects => generated 409 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:32.803200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31f1cd-5df1-4f9a-9164-d7ae39dc10ff admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:32.816795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31f1cd-5df1-4f9a-9164-d7ae39dc10ff admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:32.876952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe15945-522c-4709-8418-e355e5558164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:32.877754 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe15945-522c-4709-8418-e355e5558164 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:32.878142 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe15945-522c-4709-8418-e355e5558164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:32.878518 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe15945-522c-4709-8418-e355e5558164 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:32.879526 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe15945-522c-4709-8418-e355e5558164 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:32.881700 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe15945-522c-4709-8418-e355e5558164 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:32.908995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1042/2070] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:13:32 2022] POST /identity/v3/users => generated 305 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:13:32.913938 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d96d38-6f45-4ea6-b9f4-c0e54f495030 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:32.925008 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d96d38-6f45-4ea6-b9f4-c0e54f495030 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:32.930226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a19f2e5-b9c7-4583-ae12-a43843d1bab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:32.930817 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19f2e5-b9c7-4583-ae12-a43843d1bab0 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:32.931089 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19f2e5-b9c7-4583-ae12-a43843d1bab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:32.931360 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19f2e5-b9c7-4583-ae12-a43843d1bab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:32.932009 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a19f2e5-b9c7-4583-ae12-a43843d1bab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:32.933673 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a19f2e5-b9c7-4583-ae12-a43843d1bab0 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:32.943878 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1029/2071] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:32.948965 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe15945-522c-4709-8418-e355e5558164 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:32.959383 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe15945-522c-4709-8418-e355e5558164 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:32.964179 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb20a4a-c4ed-4b5a-836f-75349f11127e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:32.964783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb20a4a-c4ed-4b5a-836f-75349f11127e admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:32.965060 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb20a4a-c4ed-4b5a-836f-75349f11127e admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:32.965389 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb20a4a-c4ed-4b5a-836f-75349f11127e admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:32.966033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb20a4a-c4ed-4b5a-836f-75349f11127e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:32.967669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb20a4a-c4ed-4b5a-836f-75349f11127e admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:32.978677 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1043/2072] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:32.983078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a19f2e5-b9c7-4583-ae12-a43843d1bab0 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:32.993708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a19f2e5-b9c7-4583-ae12-a43843d1bab0 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:32.998601 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e80b0-e6d4-4781-bc76-1337850746cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:32.999254 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e80b0-e6d4-4781-bc76-1337850746cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:32.999520 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e80b0-e6d4-4781-bc76-1337850746cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:32.999884 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e80b0-e6d4-4781-bc76-1337850746cd admin admin] PATH_INFO: `/v3/projects/85c8ca3ca18e472d910e9702c0c18a71/users/e8afefc348ba4f4eb5ef84d945007e29/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:33.015037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e80b0-e6d4-4781-bc76-1337850746cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=85c8ca3ca18e472d910e9702c0c18a71, user_id=e8afefc348ba4f4eb5ef84d945007e29, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:33.017707 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e80b0-e6d4-4781-bc76-1337850746cd admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:33.031214 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1030/2073] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:32 2022] PUT /identity/v3/projects/85c8ca3ca18e472d910e9702c0c18a71/users/e8afefc348ba4f4eb5ef84d945007e29/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:33.042015 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb090d-845d-4cd2-9dc7-93ac828d313d None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:33.042224 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb090d-845d-4cd2-9dc7-93ac828d313d None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:33.042517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb090d-845d-4cd2-9dc7-93ac828d313d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:33.072613 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-57cb090d-845d-4cd2-9dc7-93ac828d313d None None] MFA Rules not processed for user `e8afefc348ba4f4eb5ef84d945007e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:33.074254 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-57cb090d-845d-4cd2-9dc7-93ac828d313d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:33.147240 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1044/2074] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:33 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:33.152220 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e80b0-e6d4-4781-bc76-1337850746cd admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:33.218869 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e80b0-e6d4-4781-bc76-1337850746cd admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:33.224464 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e0474-3310-4800-ab8e-772f4e6cd03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:33.225038 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e0474-3310-4800-ab8e-772f4e6cd03c admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:33.225408 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e0474-3310-4800-ab8e-772f4e6cd03c admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:33.225690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e0474-3310-4800-ab8e-772f4e6cd03c admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:33.226366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e0474-3310-4800-ab8e-772f4e6cd03c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:33.228339 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e0474-3310-4800-ab8e-772f4e6cd03c admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:33.240475 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:33.277808 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1031/2075] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:13:33 2022] POST /identity/v3/projects => generated 409 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:33.282534 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb090d-845d-4cd2-9dc7-93ac828d313d None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:33.295091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb090d-845d-4cd2-9dc7-93ac828d313d None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:33.329338 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:33.361979 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1684f-daa2-41f6-a9e9-30de1fad0d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:33.362552 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1684f-daa2-41f6-a9e9-30de1fad0d69 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:33.362828 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1684f-daa2-41f6-a9e9-30de1fad0d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:33.363099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1684f-daa2-41f6-a9e9-30de1fad0d69 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:33.363752 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1684f-daa2-41f6-a9e9-30de1fad0d69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:33.365855 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1684f-daa2-41f6-a9e9-30de1fad0d69 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:33.391562 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1045/2076] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:13:33 2022] POST /identity/v3/users => generated 311 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:33.398008 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e0474-3310-4800-ab8e-772f4e6cd03c admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:33.412487 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e0474-3310-4800-ab8e-772f4e6cd03c admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:33.419060 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea8677-0578-4652-bcce-6ee3aa0ed903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:33.419672 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea8677-0578-4652-bcce-6ee3aa0ed903 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:33.419947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea8677-0578-4652-bcce-6ee3aa0ed903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:33.420237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea8677-0578-4652-bcce-6ee3aa0ed903 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:33.420861 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea8677-0578-4652-bcce-6ee3aa0ed903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:33.422792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea8677-0578-4652-bcce-6ee3aa0ed903 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:33.437517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1032/2077] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:33.441760 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1684f-daa2-41f6-a9e9-30de1fad0d69 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:33.453631 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1684f-daa2-41f6-a9e9-30de1fad0d69 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:33.458535 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e29f9b-91b7-4e2d-9ce2-108564dee720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:33.459338 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e29f9b-91b7-4e2d-9ce2-108564dee720 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:33.459732 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e29f9b-91b7-4e2d-9ce2-108564dee720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:33.460006 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e29f9b-91b7-4e2d-9ce2-108564dee720 admin admin] PATH_INFO: `/v3/projects/cd6d151923944528b878ef7dbcba0523/users/99effabf35a644ffbdbb255ab74d608e/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:33.475617 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e29f9b-91b7-4e2d-9ce2-108564dee720 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd6d151923944528b878ef7dbcba0523, user_id=99effabf35a644ffbdbb255ab74d608e, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:33.478335 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e29f9b-91b7-4e2d-9ce2-108564dee720 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:33.492554 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1046/2078] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:33 2022] PUT /identity/v3/projects/cd6d151923944528b878ef7dbcba0523/users/99effabf35a644ffbdbb255ab74d608e/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:33.496604 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea8677-0578-4652-bcce-6ee3aa0ed903 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:33.507081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea8677-0578-4652-bcce-6ee3aa0ed903 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:33.568722 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d8098-f10e-4dba-885c-b5c3f5b14dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:33.569280 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d8098-f10e-4dba-885c-b5c3f5b14dbc admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:33.569561 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d8098-f10e-4dba-885c-b5c3f5b14dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:33.569830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d8098-f10e-4dba-885c-b5c3f5b14dbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:33.570504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d8098-f10e-4dba-885c-b5c3f5b14dbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:33.572117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d8098-f10e-4dba-885c-b5c3f5b14dbc admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:33.582741 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1033/2079] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:33.587350 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e29f9b-91b7-4e2d-9ce2-108564dee720 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:33.598156 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e29f9b-91b7-4e2d-9ce2-108564dee720 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:33.603199 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f69ccb-8a84-4a6d-80c5-4a977ea44834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:33.603977 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f69ccb-8a84-4a6d-80c5-4a977ea44834 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:33.604256 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f69ccb-8a84-4a6d-80c5-4a977ea44834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:33.604544 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f69ccb-8a84-4a6d-80c5-4a977ea44834 admin admin] PATH_INFO: `/v3/projects/cd6d151923944528b878ef7dbcba0523/users/99effabf35a644ffbdbb255ab74d608e/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:33.606473 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f69ccb-8a84-4a6d-80c5-4a977ea44834 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd6d151923944528b878ef7dbcba0523, user_id=99effabf35a644ffbdbb255ab74d608e, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:33.609151 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f69ccb-8a84-4a6d-80c5-4a977ea44834 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:33.621547 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1047/2080] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:33 2022] PUT /identity/v3/projects/cd6d151923944528b878ef7dbcba0523/users/99effabf35a644ffbdbb255ab74d608e/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:33.633791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd97678-23e5-41b1-8a93-b935b23de7db None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:33.634139 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd97678-23e5-41b1-8a93-b935b23de7db None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:33.634506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd97678-23e5-41b1-8a93-b935b23de7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:33.664307 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-0dd97678-23e5-41b1-8a93-b935b23de7db None None] MFA Rules not processed for user `99effabf35a644ffbdbb255ab74d608e`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:33.665585 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-0dd97678-23e5-41b1-8a93-b935b23de7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:33.739436 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1034/2081] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:33 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:33.751767 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f69ccb-8a84-4a6d-80c5-4a977ea44834 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:33.753673 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-42f69ccb-8a84-4a6d-80c5-4a977ea44834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:33.818084 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f69ccb-8a84-4a6d-80c5-4a977ea44834 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:33.822201 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c346834-1909-4eda-aa80-0e046d3ed086 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:33.822779 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c346834-1909-4eda-aa80-0e046d3ed086 service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:33.823048 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c346834-1909-4eda-aa80-0e046d3ed086 service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:33.823316 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c346834-1909-4eda-aa80-0e046d3ed086 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:33.833734 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c346834-1909-4eda-aa80-0e046d3ed086 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:33.835167 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c346834-1909-4eda-aa80-0e046d3ed086 service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:33.845174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1048/2082] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:33 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:34.022008 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd97678-23e5-41b1-8a93-b935b23de7db None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:34.023516 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-0dd97678-23e5-41b1-8a93-b935b23de7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:34.036138 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd97678-23e5-41b1-8a93-b935b23de7db None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:34.040471 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801871ef-9c48-4c5d-889f-62ee956a9f8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:34.041210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801871ef-9c48-4c5d-889f-62ee956a9f8d service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:34.041424 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801871ef-9c48-4c5d-889f-62ee956a9f8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:34.041777 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801871ef-9c48-4c5d-889f-62ee956a9f8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:34.120030 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801871ef-9c48-4c5d-889f-62ee956a9f8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:34.121508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801871ef-9c48-4c5d-889f-62ee956a9f8d service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:34.125015 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:34.132080 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1035/2083] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:34 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:35.576720 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c346834-1909-4eda-aa80-0e046d3ed086 service cinder] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:35.578175 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-9c346834-1909-4eda-aa80-0e046d3ed086 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:35.646840 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c346834-1909-4eda-aa80-0e046d3ed086 service cinder] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:35.651550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec32d31-f020-409c-992f-a48400fc4c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:35.652195 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec32d31-f020-409c-992f-a48400fc4c9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:35.652463 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec32d31-f020-409c-992f-a48400fc4c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:35.652729 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec32d31-f020-409c-992f-a48400fc4c9f admin admin] PATH_INFO: `/v3/users/259bbefd7cec436188717c99e6b340e5` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:35.653869 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec32d31-f020-409c-992f-a48400fc4c9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=259bbefd7cec436188717c99e6b340e5)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:35.655789 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec32d31-f020-409c-992f-a48400fc4c9f admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:35.689771 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:35.725395 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-6ec32d31-f020-409c-992f-a48400fc4c9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '259bbefd7cec436188717c99e6b340e5'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:35.734017 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-6ec32d31-f020-409c-992f-a48400fc4c9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '259bbefd7cec436188717c99e6b340e5'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:35.747328 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-6ec32d31-f020-409c-992f-a48400fc4c9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '259bbefd7cec436188717c99e6b340e5'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:35.747975 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-6ec32d31-f020-409c-992f-a48400fc4c9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '259bbefd7cec436188717c99e6b340e5'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:35.755259 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-6ec32d31-f020-409c-992f-a48400fc4c9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '259bbefd7cec436188717c99e6b340e5'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:35.761494 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1049/2084] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:35 2022] DELETE /identity/v3/users/259bbefd7cec436188717c99e6b340e5 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:35.766177 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084af89-0f2c-44aa-bfb4-50435a95ad9b None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:35.766662 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084af89-0f2c-44aa-bfb4-50435a95ad9b None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:35.766768 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084af89-0f2c-44aa-bfb4-50435a95ad9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:35.787726 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-1084af89-0f2c-44aa-bfb4-50435a95ad9b None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:35.788945 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-1084af89-0f2c-44aa-bfb4-50435a95ad9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:35.853317 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1036/2085] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:35.863478 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec32d31-f020-409c-992f-a48400fc4c9f admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:35.864977 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-6ec32d31-f020-409c-992f-a48400fc4c9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:35.924456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec32d31-f020-409c-992f-a48400fc4c9f admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:35.928448 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181fd399-7eff-4b62-858e-5348e58e3eef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:35.929057 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fd399-7eff-4b62-858e-5348e58e3eef service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:35.929364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fd399-7eff-4b62-858e-5348e58e3eef service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:35.929627 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fd399-7eff-4b62-858e-5348e58e3eef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:35.939964 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181fd399-7eff-4b62-858e-5348e58e3eef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:35.941436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181fd399-7eff-4b62-858e-5348e58e3eef service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:35.951866 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1050/2086] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:36.238557 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084af89-0f2c-44aa-bfb4-50435a95ad9b None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:36.240722 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-1084af89-0f2c-44aa-bfb4-50435a95ad9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:36.260050 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084af89-0f2c-44aa-bfb4-50435a95ad9b None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:36.267048 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abdb801-c263-4c29-968b-19567f06d047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:36.267697 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abdb801-c263-4c29-968b-19567f06d047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:36.267881 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abdb801-c263-4c29-968b-19567f06d047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:36.268151 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abdb801-c263-4c29-968b-19567f06d047 admin admin] PATH_INFO: `/v3/projects/7010b191373145608fe85e0e89085a63` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:36.269845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abdb801-c263-4c29-968b-19567f06d047 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7010b191373145608fe85e0e89085a63)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:36.271465 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abdb801-c263-4c29-968b-19567f06d047 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:36.324298 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8abdb801-c263-4c29-968b-19567f06d047 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7010b191373145608fe85e0e89085a63', 'request_id': 'req-8abdb801-c263-4c29-968b-19567f06d047'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:36.333860 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8abdb801-c263-4c29-968b-19567f06d047 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7010b191373145608fe85e0e89085a63', 'request_id': 'req-8abdb801-c263-4c29-968b-19567f06d047'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:36.344991 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8abdb801-c263-4c29-968b-19567f06d047 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7010b191373145608fe85e0e89085a63', 'request_id': 'req-8abdb801-c263-4c29-968b-19567f06d047'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:36.356480 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8abdb801-c263-4c29-968b-19567f06d047 admin admin] The token cache is being invalidate because project 7010b191373145608fe85e0e89085a63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:36.356772 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8abdb801-c263-4c29-968b-19567f06d047 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:36.357560 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8abdb801-c263-4c29-968b-19567f06d047 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:36.359070 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1037/2087] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:36 2022] DELETE /identity/v3/projects/7010b191373145608fe85e0e89085a63 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:36.363335 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181fd399-7eff-4b62-858e-5348e58e3eef service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:36.373116 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181fd399-7eff-4b62-858e-5348e58e3eef service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:36.431649 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4960aa-199c-4296-b9f3-d28e4fc93201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:36.432299 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4960aa-199c-4296-b9f3-d28e4fc93201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:36.432569 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4960aa-199c-4296-b9f3-d28e4fc93201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:36.432871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4960aa-199c-4296-b9f3-d28e4fc93201 admin admin] PATH_INFO: `/v3/users/a0ee273924b24ef39f10e75ebeed6098` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:36.434033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4960aa-199c-4296-b9f3-d28e4fc93201 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0ee273924b24ef39f10e75ebeed6098)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:36.435929 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4960aa-199c-4296-b9f3-d28e4fc93201 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:36.484426 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-0c4960aa-199c-4296-b9f3-d28e4fc93201 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0ee273924b24ef39f10e75ebeed6098'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:36.486950 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:36.492740 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-0c4960aa-199c-4296-b9f3-d28e4fc93201 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0ee273924b24ef39f10e75ebeed6098'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:36.505956 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-0c4960aa-199c-4296-b9f3-d28e4fc93201 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0ee273924b24ef39f10e75ebeed6098'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:36.506619 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-0c4960aa-199c-4296-b9f3-d28e4fc93201 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0ee273924b24ef39f10e75ebeed6098'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:36.513906 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-0c4960aa-199c-4296-b9f3-d28e4fc93201 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0ee273924b24ef39f10e75ebeed6098'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:36.520223 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1051/2088] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:36 2022] DELETE /identity/v3/users/a0ee273924b24ef39f10e75ebeed6098 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:36.809054 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abdb801-c263-4c29-968b-19567f06d047 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:36.810686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-8abdb801-c263-4c29-968b-19567f06d047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:36.826909 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abdb801-c263-4c29-968b-19567f06d047 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:36.892965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b1735-f68a-4e1b-9c84-0f7b894b4d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:36.893758 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b1735-f68a-4e1b-9c84-0f7b894b4d20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:36.894040 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b1735-f68a-4e1b-9c84-0f7b894b4d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:36.894309 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b1735-f68a-4e1b-9c84-0f7b894b4d20 admin admin] PATH_INFO: `/v3/projects/1d2fc087f1574e24869bc01f1f49f064` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:36.895486 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b1735-f68a-4e1b-9c84-0f7b894b4d20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d2fc087f1574e24869bc01f1f49f064)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:36.897428 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b1735-f68a-4e1b-9c84-0f7b894b4d20 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:36.924000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:36.949910 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c30b1735-f68a-4e1b-9c84-0f7b894b4d20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d2fc087f1574e24869bc01f1f49f064', 'request_id': 'req-c30b1735-f68a-4e1b-9c84-0f7b894b4d20'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:36.957520 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c30b1735-f68a-4e1b-9c84-0f7b894b4d20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d2fc087f1574e24869bc01f1f49f064', 'request_id': 'req-c30b1735-f68a-4e1b-9c84-0f7b894b4d20'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:36.964396 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c30b1735-f68a-4e1b-9c84-0f7b894b4d20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d2fc087f1574e24869bc01f1f49f064', 'request_id': 'req-c30b1735-f68a-4e1b-9c84-0f7b894b4d20'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:36.975891 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c30b1735-f68a-4e1b-9c84-0f7b894b4d20 admin admin] The token cache is being invalidate because project 1d2fc087f1574e24869bc01f1f49f064 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:36.976179 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c30b1735-f68a-4e1b-9c84-0f7b894b4d20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:36.976929 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c30b1735-f68a-4e1b-9c84-0f7b894b4d20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:36.978533 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1038/2089] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:36 2022] DELETE /identity/v3/projects/1d2fc087f1574e24869bc01f1f49f064 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:36.982796 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4960aa-199c-4296-b9f3-d28e4fc93201 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:36.984462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-0c4960aa-199c-4296-b9f3-d28e4fc93201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:36.998405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4960aa-199c-4296-b9f3-d28e4fc93201 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:37.059003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a07a80-596c-4fa5-b500-7fc775af44a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:37.059629 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a07a80-596c-4fa5-b500-7fc775af44a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:37.059902 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a07a80-596c-4fa5-b500-7fc775af44a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:37.060171 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a07a80-596c-4fa5-b500-7fc775af44a7 admin admin] PATH_INFO: `/v3/users/bb7efb83d5e645249213bb11bb240e95` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:37.061360 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a07a80-596c-4fa5-b500-7fc775af44a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb7efb83d5e645249213bb11bb240e95)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:37.063281 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a07a80-596c-4fa5-b500-7fc775af44a7 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:37.112479 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-34a07a80-596c-4fa5-b500-7fc775af44a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb7efb83d5e645249213bb11bb240e95'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:37.120658 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-34a07a80-596c-4fa5-b500-7fc775af44a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb7efb83d5e645249213bb11bb240e95'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:37.133016 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-34a07a80-596c-4fa5-b500-7fc775af44a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb7efb83d5e645249213bb11bb240e95'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:37.133741 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-34a07a80-596c-4fa5-b500-7fc775af44a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb7efb83d5e645249213bb11bb240e95'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:37.140716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-34a07a80-596c-4fa5-b500-7fc775af44a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb7efb83d5e645249213bb11bb240e95'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:37.147165 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1052/2090] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:36 2022] DELETE /identity/v3/users/bb7efb83d5e645249213bb11bb240e95 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:37.430174 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b1735-f68a-4e1b-9c84-0f7b894b4d20 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:37.432085 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c30b1735-f68a-4e1b-9c84-0f7b894b4d20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:37.446045 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b1735-f68a-4e1b-9c84-0f7b894b4d20 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:37.513172 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf9a82-3689-4a81-a6e0-155c55590e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:37.513822 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf9a82-3689-4a81-a6e0-155c55590e4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:37.514093 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf9a82-3689-4a81-a6e0-155c55590e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:37.514418 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf9a82-3689-4a81-a6e0-155c55590e4f admin admin] PATH_INFO: `/v3/projects/881128a82b8d4fb1a0b500801ee5397e` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:37.515550 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bf9a82-3689-4a81-a6e0-155c55590e4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=881128a82b8d4fb1a0b500801ee5397e)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:37.517489 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bf9a82-3689-4a81-a6e0-155c55590e4f admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:37.572013 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d3bf9a82-3689-4a81-a6e0-155c55590e4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '881128a82b8d4fb1a0b500801ee5397e', 'request_id': 'req-d3bf9a82-3689-4a81-a6e0-155c55590e4f'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:37.582504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d3bf9a82-3689-4a81-a6e0-155c55590e4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '881128a82b8d4fb1a0b500801ee5397e', 'request_id': 'req-d3bf9a82-3689-4a81-a6e0-155c55590e4f'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:37.589528 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d3bf9a82-3689-4a81-a6e0-155c55590e4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '881128a82b8d4fb1a0b500801ee5397e', 'request_id': 'req-d3bf9a82-3689-4a81-a6e0-155c55590e4f'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:37.602383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d3bf9a82-3689-4a81-a6e0-155c55590e4f admin admin] The token cache is being invalidate because project 881128a82b8d4fb1a0b500801ee5397e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:37.602834 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d3bf9a82-3689-4a81-a6e0-155c55590e4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:37.603671 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d3bf9a82-3689-4a81-a6e0-155c55590e4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:37.605549 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1039/2091] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:37 2022] DELETE /identity/v3/projects/881128a82b8d4fb1a0b500801ee5397e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:37.610422 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91864488-2d9e-4a54-b836-7d631c9994c3 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:37.610706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91864488-2d9e-4a54-b836-7d631c9994c3 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:37.611001 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91864488-2d9e-4a54-b836-7d631c9994c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:37.632458 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-91864488-2d9e-4a54-b836-7d631c9994c3 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:37.633702 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-91864488-2d9e-4a54-b836-7d631c9994c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:37.677809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:37.697421 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1053/2092] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:37.706958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be978722-c932-4031-8156-18be83ae1449 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:37.707322 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be978722-c932-4031-8156-18be83ae1449 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:37.707621 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be978722-c932-4031-8156-18be83ae1449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:37.729983 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-be978722-c932-4031-8156-18be83ae1449 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:37.731231 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-be978722-c932-4031-8156-18be83ae1449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:37.739689 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1040/2093] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:37.744188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91864488-2d9e-4a54-b836-7d631c9994c3 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:37.754205 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91864488-2d9e-4a54-b836-7d631c9994c3 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:37.759202 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e88fb5-8573-4670-989d-ec7a95756f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:37.759810 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e88fb5-8573-4670-989d-ec7a95756f48 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:37.760089 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e88fb5-8573-4670-989d-ec7a95756f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:37.760354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e88fb5-8573-4670-989d-ec7a95756f48 admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:37.761011 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e88fb5-8573-4670-989d-ec7a95756f48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:13:37.761332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e88fb5-8573-4670-989d-ec7a95756f48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:37.762967 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e88fb5-8573-4670-989d-ec7a95756f48 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:37.775336 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1054/2094] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:13:37.780103 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be978722-c932-4031-8156-18be83ae1449 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:37.790006 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be978722-c932-4031-8156-18be83ae1449 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:37.795698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6d7d7-9186-4b5f-9a80-e16a2aa6820b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:37.796312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6d7d7-9186-4b5f-9a80-e16a2aa6820b admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:37.796588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6d7d7-9186-4b5f-9a80-e16a2aa6820b admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:37.796856 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6d7d7-9186-4b5f-9a80-e16a2aa6820b admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:37.797556 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6d7d7-9186-4b5f-9a80-e16a2aa6820b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:37.799509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6d7d7-9186-4b5f-9a80-e16a2aa6820b admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:37.831503 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1041/2095] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:13:37 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:13:37.836092 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e88fb5-8573-4670-989d-ec7a95756f48 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:37.847158 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e88fb5-8573-4670-989d-ec7a95756f48 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:37.911748 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbfed09-541f-4690-950d-a9985297266e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:37.912351 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbfed09-541f-4690-950d-a9985297266e admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:37.912624 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbfed09-541f-4690-950d-a9985297266e admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:37.912893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbfed09-541f-4690-950d-a9985297266e admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:37.913562 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbfed09-541f-4690-950d-a9985297266e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:37.915523 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbfed09-541f-4690-950d-a9985297266e admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:37.938736 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1055/2096] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:13:37 2022] POST /identity/v3/users => generated 285 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:37.942923 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6d7d7-9186-4b5f-9a80-e16a2aa6820b admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:37.954520 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6d7d7-9186-4b5f-9a80-e16a2aa6820b admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:37.960381 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b0ef7-0c0c-4dd9-9e8a-93ad6aff0828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:37.961028 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b0ef7-0c0c-4dd9-9e8a-93ad6aff0828 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:37.961345 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b0ef7-0c0c-4dd9-9e8a-93ad6aff0828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:37.961612 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b0ef7-0c0c-4dd9-9e8a-93ad6aff0828 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:37.962274 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b0ef7-0c0c-4dd9-9e8a-93ad6aff0828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:37.963940 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b0ef7-0c0c-4dd9-9e8a-93ad6aff0828 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:37.975921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1042/2097] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:37.980038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbfed09-541f-4690-950d-a9985297266e admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:37.991426 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbfed09-541f-4690-950d-a9985297266e admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:37.999473 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324b46f-97f8-4113-8fdc-0a64a5f08724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:38.000355 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324b46f-97f8-4113-8fdc-0a64a5f08724 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:38.000826 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324b46f-97f8-4113-8fdc-0a64a5f08724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:38.001154 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324b46f-97f8-4113-8fdc-0a64a5f08724 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:38.001929 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9324b46f-97f8-4113-8fdc-0a64a5f08724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:38.004075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9324b46f-97f8-4113-8fdc-0a64a5f08724 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:38.017981 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1056/2098] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:38.022414 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b0ef7-0c0c-4dd9-9e8a-93ad6aff0828 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:38.027810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:38.035075 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b0ef7-0c0c-4dd9-9e8a-93ad6aff0828 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:38.041619 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6404d-6742-45d3-af53-6d57800eecb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:38.042304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6404d-6742-45d3-af53-6d57800eecb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:38.042769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6404d-6742-45d3-af53-6d57800eecb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:38.043220 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6404d-6742-45d3-af53-6d57800eecb0 admin admin] PATH_INFO: `/v3/projects/554e4c2a862b447f859321337f3135e6/users/e8aafc81a7754e839b45deef6c16cffe/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:38.060101 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6404d-6742-45d3-af53-6d57800eecb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=554e4c2a862b447f859321337f3135e6, user_id=e8aafc81a7754e839b45deef6c16cffe, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:38.062806 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6404d-6742-45d3-af53-6d57800eecb0 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:38.078140 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1043/2099] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:38 2022] PUT /identity/v3/projects/554e4c2a862b447f859321337f3135e6/users/e8aafc81a7754e839b45deef6c16cffe/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:13:38.088788 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f59e3a-d6bf-40b7-ad0e-908111b41cce None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:38.089172 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f59e3a-d6bf-40b7-ad0e-908111b41cce None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:38.089513 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f59e3a-d6bf-40b7-ad0e-908111b41cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:38.130865 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-90f59e3a-d6bf-40b7-ad0e-908111b41cce None None] MFA Rules not processed for user `e8aafc81a7754e839b45deef6c16cffe`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:38.132447 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-90f59e3a-d6bf-40b7-ad0e-908111b41cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:38.214233 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1057/2100] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:13:38.226274 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6404d-6742-45d3-af53-6d57800eecb0 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:38.228176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-e1f6404d-6742-45d3-af53-6d57800eecb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:38.294918 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6404d-6742-45d3-af53-6d57800eecb0 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:38.299340 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a55ff28-75de-4584-bccd-523ecb87e979 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fe5293447346eaafd16818e5bd12f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:38.299949 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55ff28-75de-4584-bccd-523ecb87e979 service glance] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:38.300217 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55ff28-75de-4584-bccd-523ecb87e979 service glance] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:38.300481 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55ff28-75de-4584-bccd-523ecb87e979 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:38.311096 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a55ff28-75de-4584-bccd-523ecb87e979 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:38.312707 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a55ff28-75de-4584-bccd-523ecb87e979 service glance] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:38.322662 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1044/2101] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:43.924483 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f59e3a-d6bf-40b7-ad0e-908111b41cce None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:43.927461 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-90f59e3a-d6bf-40b7-ad0e-908111b41cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:44.021396 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:44.048234 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f59e3a-d6bf-40b7-ad0e-908111b41cce None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:44.053716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e9c42-86e0-486a-a8c4-b63440f0b8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:44.054676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e9c42-86e0-486a-a8c4-b63440f0b8e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:44.055087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e9c42-86e0-486a-a8c4-b63440f0b8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:44.055406 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e9c42-86e0-486a-a8c4-b63440f0b8e8 admin admin] PATH_INFO: `/v3/users/dc5042d69e604dc999fc9100897e6b82` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:44.056827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e9c42-86e0-486a-a8c4-b63440f0b8e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc5042d69e604dc999fc9100897e6b82)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:44.058977 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e9c42-86e0-486a-a8c4-b63440f0b8e8 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:44.138761 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-524e9c42-86e0-486a-a8c4-b63440f0b8e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc5042d69e604dc999fc9100897e6b82'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:44.150433 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-524e9c42-86e0-486a-a8c4-b63440f0b8e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc5042d69e604dc999fc9100897e6b82'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:44.173920 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-524e9c42-86e0-486a-a8c4-b63440f0b8e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc5042d69e604dc999fc9100897e6b82'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:44.175285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-524e9c42-86e0-486a-a8c4-b63440f0b8e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc5042d69e604dc999fc9100897e6b82'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:44.200498 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-524e9c42-86e0-486a-a8c4-b63440f0b8e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc5042d69e604dc999fc9100897e6b82'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:44.209585 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1058/2102] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:43 2022] DELETE /identity/v3/users/dc5042d69e604dc999fc9100897e6b82 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:44.215918 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d540d66-9db8-4093-adae-6b30abf20b78 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:44.216777 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d540d66-9db8-4093-adae-6b30abf20b78 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:44.216777 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d540d66-9db8-4093-adae-6b30abf20b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:44.243634 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-8d540d66-9db8-4093-adae-6b30abf20b78 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:44.244943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-8d540d66-9db8-4093-adae-6b30abf20b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:44.344465 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1045/2103] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:13:44.359399 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e9c42-86e0-486a-a8c4-b63440f0b8e8 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:44.361945 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-524e9c42-86e0-486a-a8c4-b63440f0b8e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:44.462773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e9c42-86e0-486a-a8c4-b63440f0b8e8 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:44.469219 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5785a9a8-8aa5-4fe2-a6e9-a4c7f902b863 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:44.470225 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785a9a8-8aa5-4fe2-a6e9-a4c7f902b863 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:44.470668 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785a9a8-8aa5-4fe2-a6e9-a4c7f902b863 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:44.471096 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785a9a8-8aa5-4fe2-a6e9-a4c7f902b863 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:44.487001 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5785a9a8-8aa5-4fe2-a6e9-a4c7f902b863 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:44.489895 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5785a9a8-8aa5-4fe2-a6e9-a4c7f902b863 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:44.508300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1059/2104] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:44.950572 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d540d66-9db8-4093-adae-6b30abf20b78 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:44.958672 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-8d540d66-9db8-4093-adae-6b30abf20b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:44.982290 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d540d66-9db8-4093-adae-6b30abf20b78 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:44.998380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5785a9a8-8aa5-4fe2-a6e9-a4c7f902b863 service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:45.002037 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-5785a9a8-8aa5-4fe2-a6e9-a4c7f902b863 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:45.016884 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4066e4-5d9b-4817-b3cd-b601809eed54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:45.020350 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4066e4-5d9b-4817-b3cd-b601809eed54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:45.020640 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4066e4-5d9b-4817-b3cd-b601809eed54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:45.020871 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4066e4-5d9b-4817-b3cd-b601809eed54 admin admin] PATH_INFO: `/v3/users/e8afefc348ba4f4eb5ef84d945007e29` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:45.022825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4066e4-5d9b-4817-b3cd-b601809eed54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8afefc348ba4f4eb5ef84d945007e29)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:45.042090 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4066e4-5d9b-4817-b3cd-b601809eed54 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:45.067839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5785a9a8-8aa5-4fe2-a6e9-a4c7f902b863 service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:45.085626 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eff852-5c5f-473c-b08a-abd98574da74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:45.088448 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eff852-5c5f-473c-b08a-abd98574da74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:45.089316 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eff852-5c5f-473c-b08a-abd98574da74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:45.090253 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eff852-5c5f-473c-b08a-abd98574da74 admin admin] PATH_INFO: `/v3/projects/d608a9b9311d47bc8933913d4f88a26e` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:45.092808 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eff852-5c5f-473c-b08a-abd98574da74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d608a9b9311d47bc8933913d4f88a26e)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:45.097109 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eff852-5c5f-473c-b08a-abd98574da74 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:45.175849 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-ac4066e4-5d9b-4817-b3cd-b601809eed54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8afefc348ba4f4eb5ef84d945007e29'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:45.194119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-ac4066e4-5d9b-4817-b3cd-b601809eed54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8afefc348ba4f4eb5ef84d945007e29'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:45.279074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-ac4066e4-5d9b-4817-b3cd-b601809eed54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8afefc348ba4f4eb5ef84d945007e29'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:45.283691 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-ac4066e4-5d9b-4817-b3cd-b601809eed54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8afefc348ba4f4eb5ef84d945007e29'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:45.303933 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:45.320558 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-ac4066e4-5d9b-4817-b3cd-b601809eed54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8afefc348ba4f4eb5ef84d945007e29'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:45.334909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-26eff852-5c5f-473c-b08a-abd98574da74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd608a9b9311d47bc8933913d4f88a26e', 'request_id': 'req-26eff852-5c5f-473c-b08a-abd98574da74'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:45.335903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1046/2105] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:44 2022] DELETE /identity/v3/users/e8afefc348ba4f4eb5ef84d945007e29 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:45.338231 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4066e4-5d9b-4817-b3cd-b601809eed54 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:45.340576 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-ac4066e4-5d9b-4817-b3cd-b601809eed54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:45.369094 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-26eff852-5c5f-473c-b08a-abd98574da74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd608a9b9311d47bc8933913d4f88a26e', 'request_id': 'req-26eff852-5c5f-473c-b08a-abd98574da74'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:45.374708 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:45.430173 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-26eff852-5c5f-473c-b08a-abd98574da74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd608a9b9311d47bc8933913d4f88a26e', 'request_id': 'req-26eff852-5c5f-473c-b08a-abd98574da74'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:45.479160 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-26eff852-5c5f-473c-b08a-abd98574da74 admin admin] The token cache is being invalidate because project d608a9b9311d47bc8933913d4f88a26e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:45.479897 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-26eff852-5c5f-473c-b08a-abd98574da74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:45.482083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-26eff852-5c5f-473c-b08a-abd98574da74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:45.485433 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1060/2106] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:44 2022] DELETE /identity/v3/projects/d608a9b9311d47bc8933913d4f88a26e => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:45.488000 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff7579-815c-463c-bb99-d693fd8b3f7b None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:45.489022 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff7579-815c-463c-bb99-d693fd8b3f7b None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:45.490759 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff7579-815c-463c-bb99-d693fd8b3f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:45.553788 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-4bff7579-815c-463c-bb99-d693fd8b3f7b None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:45.558310 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-4bff7579-815c-463c-bb99-d693fd8b3f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:45.588620 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4066e4-5d9b-4817-b3cd-b601809eed54 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:45.594311 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a23e4-56be-444b-87ab-34e85c5206fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:45.594956 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a23e4-56be-444b-87ab-34e85c5206fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:45.595245 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a23e4-56be-444b-87ab-34e85c5206fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:45.595525 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a23e4-56be-444b-87ab-34e85c5206fc admin admin] PATH_INFO: `/v3/users/8f684613fb624c1c9ae85d34c3c2239a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:45.596817 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a23e4-56be-444b-87ab-34e85c5206fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f684613fb624c1c9ae85d34c3c2239a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:45.598896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a23e4-56be-444b-87ab-34e85c5206fc admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:45.679253 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-ff5a23e4-56be-444b-87ab-34e85c5206fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f684613fb624c1c9ae85d34c3c2239a'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:45.685578 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1061/2107] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:45.687820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62393e-6c85-46a9-bcc3-dfedc2dbf263 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:45.688361 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62393e-6c85-46a9-bcc3-dfedc2dbf263 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:45.689161 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62393e-6c85-46a9-bcc3-dfedc2dbf263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:45.693430 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-ff5a23e4-56be-444b-87ab-34e85c5206fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f684613fb624c1c9ae85d34c3c2239a'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:45.719754 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-ff5a23e4-56be-444b-87ab-34e85c5206fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f684613fb624c1c9ae85d34c3c2239a'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:45.720893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-ff5a23e4-56be-444b-87ab-34e85c5206fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f684613fb624c1c9ae85d34c3c2239a'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:45.725474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-5e62393e-6c85-46a9-bcc3-dfedc2dbf263 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:45.727003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-5e62393e-6c85-46a9-bcc3-dfedc2dbf263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:45.735954 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-ff5a23e4-56be-444b-87ab-34e85c5206fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f684613fb624c1c9ae85d34c3c2239a'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:45.737561 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1062/2108] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:45.748873 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1047/2109] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:45 2022] DELETE /identity/v3/users/8f684613fb624c1c9ae85d34c3c2239a => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:45.752733 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62393e-6c85-46a9-bcc3-dfedc2dbf263 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:45.756152 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486beef1-34d7-4496-9e7a-fa99cb5be913 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:45.756383 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-5e62393e-6c85-46a9-bcc3-dfedc2dbf263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:45.756646 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486beef1-34d7-4496-9e7a-fa99cb5be913 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:45.757100 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486beef1-34d7-4496-9e7a-fa99cb5be913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:45.793524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-486beef1-34d7-4496-9e7a-fa99cb5be913 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:45.795625 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-486beef1-34d7-4496-9e7a-fa99cb5be913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:45.857389 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62393e-6c85-46a9-bcc3-dfedc2dbf263 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:45.912593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1048/2110] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:13:45.915991 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18a784-a06d-46fb-a401-9346a8ccfa50 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:45.916744 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18a784-a06d-46fb-a401-9346a8ccfa50 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:45.917619 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18a784-a06d-46fb-a401-9346a8ccfa50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:45.933692 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836a1c3-e2a6-4f98-b6fc-21eacd7c31ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:45.934474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836a1c3-e2a6-4f98-b6fc-21eacd7c31ea service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:45.934811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836a1c3-e2a6-4f98-b6fc-21eacd7c31ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:45.935269 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836a1c3-e2a6-4f98-b6fc-21eacd7c31ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:45.938423 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-8836a1c3-e2a6-4f98-b6fc-21eacd7c31ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:45.956710 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8836a1c3-e2a6-4f98-b6fc-21eacd7c31ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:45.956970 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-bc18a784-a06d-46fb-a401-9346a8ccfa50 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:45.958565 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8836a1c3-e2a6-4f98-b6fc-21eacd7c31ea service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:45.958831 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-bc18a784-a06d-46fb-a401-9346a8ccfa50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:45.971318 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1063/2111] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:45.972489 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836a1c3-e2a6-4f98-b6fc-21eacd7c31ea service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:45.979059 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1049/2112] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:45.989466 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836a1c3-e2a6-4f98-b6fc-21eacd7c31ea service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:45.997122 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7dc3ef-2819-4d3c-b917-9a7c8be46fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:45.998099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7dc3ef-2819-4d3c-b917-9a7c8be46fda admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:45.998545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7dc3ef-2819-4d3c-b917-9a7c8be46fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:45.999086 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7dc3ef-2819-4d3c-b917-9a7c8be46fda admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:46.000074 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7dc3ef-2819-4d3c-b917-9a7c8be46fda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:13:46.000575 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7dc3ef-2819-4d3c-b917-9a7c8be46fda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:46.003144 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18a784-a06d-46fb-a401-9346a8ccfa50 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:46.003363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7dc3ef-2819-4d3c-b917-9a7c8be46fda admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:46.025388 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1064/2113] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:13:46.028013 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18a784-a06d-46fb-a401-9346a8ccfa50 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:46.032518 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7dc3ef-2819-4d3c-b917-9a7c8be46fda admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:46.033256 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b3c98-b3e9-4bbd-bcc4-ca5b84493583 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:46.034264 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b3c98-b3e9-4bbd-bcc4-ca5b84493583 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:46.034721 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b3c98-b3e9-4bbd-bcc4-ca5b84493583 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:46.035171 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b3c98-b3e9-4bbd-bcc4-ca5b84493583 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:46.045752 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:46.051089 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b3c98-b3e9-4bbd-bcc4-ca5b84493583 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:46.051731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7dc3ef-2819-4d3c-b917-9a7c8be46fda admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:46.053616 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b3c98-b3e9-4bbd-bcc4-ca5b84493583 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:46.057434 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1264b4-bf5a-4601-a31d-b6d9a400629c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:46.058073 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1264b4-bf5a-4601-a31d-b6d9a400629c admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:46.058404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1264b4-bf5a-4601-a31d-b6d9a400629c admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:46.059449 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1264b4-bf5a-4601-a31d-b6d9a400629c admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:46.060526 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1264b4-bf5a-4601-a31d-b6d9a400629c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:46.062047 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1264b4-bf5a-4601-a31d-b6d9a400629c admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:46.096467 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1050/2114] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:46.104978 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1065/2115] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:13:46 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:13:46.108548 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b3c98-b3e9-4bbd-bcc4-ca5b84493583 service neutron] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:46.125480 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b3c98-b3e9-4bbd-bcc4-ca5b84493583 service neutron] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:46.209506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906958c-35a6-4687-8dca-ad0efd8c2e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:46.209832 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906958c-35a6-4687-8dca-ad0efd8c2e46 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:46.210111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906958c-35a6-4687-8dca-ad0efd8c2e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:46.210378 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906958c-35a6-4687-8dca-ad0efd8c2e46 admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:46.211191 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6906958c-35a6-4687-8dca-ad0efd8c2e46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:46.214100 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6906958c-35a6-4687-8dca-ad0efd8c2e46 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:46.246726 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1051/2116] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:13:46 2022] POST /identity/v3/users => generated 287 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:46.256381 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1264b4-bf5a-4601-a31d-b6d9a400629c admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:46.269372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:46.278979 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1264b4-bf5a-4601-a31d-b6d9a400629c admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:46.293193 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f92bf6-1eb9-47fb-8b17-c5e87bb8a9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:46.294323 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f92bf6-1eb9-47fb-8b17-c5e87bb8a9fd admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:46.294869 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f92bf6-1eb9-47fb-8b17-c5e87bb8a9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:46.295406 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f92bf6-1eb9-47fb-8b17-c5e87bb8a9fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:46.296386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f92bf6-1eb9-47fb-8b17-c5e87bb8a9fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:46.299644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f92bf6-1eb9-47fb-8b17-c5e87bb8a9fd admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:46.319599 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1066/2117] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:13:46.321606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906958c-35a6-4687-8dca-ad0efd8c2e46 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:46.334018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906958c-35a6-4687-8dca-ad0efd8c2e46 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:46.339173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297a5fe-7f32-43a7-9f9c-fbba7b8a2e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:46.339807 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297a5fe-7f32-43a7-9f9c-fbba7b8a2e4f admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:46.340086 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297a5fe-7f32-43a7-9f9c-fbba7b8a2e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:46.340352 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297a5fe-7f32-43a7-9f9c-fbba7b8a2e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:46.341012 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297a5fe-7f32-43a7-9f9c-fbba7b8a2e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:46.342716 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297a5fe-7f32-43a7-9f9c-fbba7b8a2e4f admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:46.354398 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1052/2118] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:46.359928 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f92bf6-1eb9-47fb-8b17-c5e87bb8a9fd admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:46.372553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f92bf6-1eb9-47fb-8b17-c5e87bb8a9fd admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:46.378330 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a7f07-f04e-45d3-8a3b-bcd4fe4e328c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:46.379155 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a7f07-f04e-45d3-8a3b-bcd4fe4e328c admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:46.379501 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a7f07-f04e-45d3-8a3b-bcd4fe4e328c admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:46.379837 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a7f07-f04e-45d3-8a3b-bcd4fe4e328c admin admin] PATH_INFO: `/v3/projects/3292c61ee25f4a5c892feb07fb8c344e/users/a7e0c489e68646afaf64c8cd0f6a10ed/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:46.409429 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a7f07-f04e-45d3-8a3b-bcd4fe4e328c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3292c61ee25f4a5c892feb07fb8c344e, user_id=a7e0c489e68646afaf64c8cd0f6a10ed, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:46.413225 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a7f07-f04e-45d3-8a3b-bcd4fe4e328c admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:46.443339 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1067/2119] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:46 2022] PUT /identity/v3/projects/3292c61ee25f4a5c892feb07fb8c344e/users/a7e0c489e68646afaf64c8cd0f6a10ed/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:46.463779 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770ad9e-94e7-4ee4-bcdd-29d5c7af75b7 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:46.464277 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770ad9e-94e7-4ee4-bcdd-29d5c7af75b7 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:46.464768 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770ad9e-94e7-4ee4-bcdd-29d5c7af75b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:46.533452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-6770ad9e-94e7-4ee4-bcdd-29d5c7af75b7 None None] MFA Rules not processed for user `a7e0c489e68646afaf64c8cd0f6a10ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:46.533452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-6770ad9e-94e7-4ee4-bcdd-29d5c7af75b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:46.725248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1053/2120] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:46.738608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a7f07-f04e-45d3-8a3b-bcd4fe4e328c admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:46.741344 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-922a7f07-f04e-45d3-8a3b-bcd4fe4e328c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:46.869254 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a7f07-f04e-45d3-8a3b-bcd4fe4e328c admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:46.869254 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a564e44-8bbb-44d3-84de-cbdb7e58b38a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:46.869254 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a564e44-8bbb-44d3-84de-cbdb7e58b38a service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:46.869254 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a564e44-8bbb-44d3-84de-cbdb7e58b38a service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:46.869881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a564e44-8bbb-44d3-84de-cbdb7e58b38a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:46.874577 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a564e44-8bbb-44d3-84de-cbdb7e58b38a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:46.881577 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a564e44-8bbb-44d3-84de-cbdb7e58b38a service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:46.904802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770ad9e-94e7-4ee4-bcdd-29d5c7af75b7 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:46.906743 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-6770ad9e-94e7-4ee4-bcdd-29d5c7af75b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:46.911705 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1068/2121] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:46.939349 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a564e44-8bbb-44d3-84de-cbdb7e58b38a service cinder] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:46.941609 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-2a564e44-8bbb-44d3-84de-cbdb7e58b38a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:47.144800 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770ad9e-94e7-4ee4-bcdd-29d5c7af75b7 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:47.155866 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57dbbd9-2b5f-413f-bfb8-9ff52a631627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:47.156816 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57dbbd9-2b5f-413f-bfb8-9ff52a631627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:47.157422 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57dbbd9-2b5f-413f-bfb8-9ff52a631627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:47.157770 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57dbbd9-2b5f-413f-bfb8-9ff52a631627 admin admin] PATH_INFO: `/v3/projects/e007facf43ab4107a24d68bfeed2e716` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:47.160885 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a564e44-8bbb-44d3-84de-cbdb7e58b38a service cinder] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:47.161716 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57dbbd9-2b5f-413f-bfb8-9ff52a631627 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e007facf43ab4107a24d68bfeed2e716)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:47.165291 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57dbbd9-2b5f-413f-bfb8-9ff52a631627 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:47.170106 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:47.172993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26bbd3-1e43-4c65-9a32-6fc60b4279f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:47.174057 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26bbd3-1e43-4c65-9a32-6fc60b4279f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:47.174392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26bbd3-1e43-4c65-9a32-6fc60b4279f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:47.174840 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26bbd3-1e43-4c65-9a32-6fc60b4279f9 admin admin] PATH_INFO: `/v3/projects/85c8ca3ca18e472d910e9702c0c18a71` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:47.178594 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f26bbd3-1e43-4c65-9a32-6fc60b4279f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85c8ca3ca18e472d910e9702c0c18a71)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:47.182208 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f26bbd3-1e43-4c65-9a32-6fc60b4279f9 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:47.261701 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-8f26bbd3-1e43-4c65-9a32-6fc60b4279f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85c8ca3ca18e472d910e9702c0c18a71', 'request_id': 'req-8f26bbd3-1e43-4c65-9a32-6fc60b4279f9'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:47.271923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-8f26bbd3-1e43-4c65-9a32-6fc60b4279f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85c8ca3ca18e472d910e9702c0c18a71', 'request_id': 'req-8f26bbd3-1e43-4c65-9a32-6fc60b4279f9'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:47.279706 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-a57dbbd9-2b5f-413f-bfb8-9ff52a631627 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e007facf43ab4107a24d68bfeed2e716', 'request_id': 'req-a57dbbd9-2b5f-413f-bfb8-9ff52a631627'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:47.283287 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-8f26bbd3-1e43-4c65-9a32-6fc60b4279f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85c8ca3ca18e472d910e9702c0c18a71', 'request_id': 'req-8f26bbd3-1e43-4c65-9a32-6fc60b4279f9'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:47.294823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-a57dbbd9-2b5f-413f-bfb8-9ff52a631627 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e007facf43ab4107a24d68bfeed2e716', 'request_id': 'req-a57dbbd9-2b5f-413f-bfb8-9ff52a631627'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:47.302991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-8f26bbd3-1e43-4c65-9a32-6fc60b4279f9 admin admin] The token cache is being invalidate because project 85c8ca3ca18e472d910e9702c0c18a71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:47.303555 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-8f26bbd3-1e43-4c65-9a32-6fc60b4279f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:47.304339 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-8f26bbd3-1e43-4c65-9a32-6fc60b4279f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:47.306347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-a57dbbd9-2b5f-413f-bfb8-9ff52a631627 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e007facf43ab4107a24d68bfeed2e716', 'request_id': 'req-a57dbbd9-2b5f-413f-bfb8-9ff52a631627'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:47.307407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1069/2122] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:46 2022] DELETE /identity/v3/projects/85c8ca3ca18e472d910e9702c0c18a71 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:13:47.311386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26bbd3-1e43-4c65-9a32-6fc60b4279f9 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:47.312859 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-8f26bbd3-1e43-4c65-9a32-6fc60b4279f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:47.319068 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:47.336300 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-a57dbbd9-2b5f-413f-bfb8-9ff52a631627 admin admin] The token cache is being invalidate because project e007facf43ab4107a24d68bfeed2e716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:47.337212 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-a57dbbd9-2b5f-413f-bfb8-9ff52a631627 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:47.337771 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-a57dbbd9-2b5f-413f-bfb8-9ff52a631627 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:47.340483 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1054/2123] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:46 2022] DELETE /identity/v3/projects/e007facf43ab4107a24d68bfeed2e716 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:47.484860 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26bbd3-1e43-4c65-9a32-6fc60b4279f9 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:47.594816 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1766860-f726-498e-9c27-e9e9adb0f719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:47.595884 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1766860-f726-498e-9c27-e9e9adb0f719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:47.596372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1766860-f726-498e-9c27-e9e9adb0f719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:47.596845 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1766860-f726-498e-9c27-e9e9adb0f719 admin admin] PATH_INFO: `/v3/users/99effabf35a644ffbdbb255ab74d608e` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:47.598853 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1766860-f726-498e-9c27-e9e9adb0f719 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99effabf35a644ffbdbb255ab74d608e)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:47.603554 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1766860-f726-498e-9c27-e9e9adb0f719 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:47.661959 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e1766860-f726-498e-9c27-e9e9adb0f719 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99effabf35a644ffbdbb255ab74d608e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:47.671299 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e1766860-f726-498e-9c27-e9e9adb0f719 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99effabf35a644ffbdbb255ab74d608e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:47.688272 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e1766860-f726-498e-9c27-e9e9adb0f719 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99effabf35a644ffbdbb255ab74d608e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:47.689087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e1766860-f726-498e-9c27-e9e9adb0f719 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99effabf35a644ffbdbb255ab74d608e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:47.696926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e1766860-f726-498e-9c27-e9e9adb0f719 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99effabf35a644ffbdbb255ab74d608e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:47.709344 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1070/2124] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:47 2022] DELETE /identity/v3/users/99effabf35a644ffbdbb255ab74d608e => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:13:48.094553 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57dbbd9-2b5f-413f-bfb8-9ff52a631627 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:48.096141 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-a57dbbd9-2b5f-413f-bfb8-9ff52a631627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:48.118836 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57dbbd9-2b5f-413f-bfb8-9ff52a631627 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:48.239022 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272c7be-b84e-458d-a452-57975c7622ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:48.239739 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272c7be-b84e-458d-a452-57975c7622ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:48.240190 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272c7be-b84e-458d-a452-57975c7622ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:48.240467 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272c7be-b84e-458d-a452-57975c7622ee admin admin] PATH_INFO: `/v3/projects/cd6d151923944528b878ef7dbcba0523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:48.241895 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272c7be-b84e-458d-a452-57975c7622ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd6d151923944528b878ef7dbcba0523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:48.244004 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272c7be-b84e-458d-a452-57975c7622ee admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:48.325768 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c272c7be-b84e-458d-a452-57975c7622ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd6d151923944528b878ef7dbcba0523', 'request_id': 'req-c272c7be-b84e-458d-a452-57975c7622ee'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:48.336478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c272c7be-b84e-458d-a452-57975c7622ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd6d151923944528b878ef7dbcba0523', 'request_id': 'req-c272c7be-b84e-458d-a452-57975c7622ee'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:48.349676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c272c7be-b84e-458d-a452-57975c7622ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd6d151923944528b878ef7dbcba0523', 'request_id': 'req-c272c7be-b84e-458d-a452-57975c7622ee'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:48.368414 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c272c7be-b84e-458d-a452-57975c7622ee admin admin] The token cache is being invalidate because project cd6d151923944528b878ef7dbcba0523 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:48.368900 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c272c7be-b84e-458d-a452-57975c7622ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:48.369670 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c272c7be-b84e-458d-a452-57975c7622ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:48.371286 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1055/2125] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:48 2022] DELETE /identity/v3/projects/cd6d151923944528b878ef7dbcba0523 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:48.376575 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e130b-af2e-4deb-bd17-a2c037f6def0 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:48.377392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e130b-af2e-4deb-bd17-a2c037f6def0 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:48.377392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e130b-af2e-4deb-bd17-a2c037f6def0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:48.397302 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-726e130b-af2e-4deb-bd17-a2c037f6def0 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:48.398524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-726e130b-af2e-4deb-bd17-a2c037f6def0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:48.428399 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:48.470978 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1071/2126] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:48.481307 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01a608-171b-43e6-b377-f56bff2a36a4 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:48.482130 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01a608-171b-43e6-b377-f56bff2a36a4 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:48.482295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01a608-171b-43e6-b377-f56bff2a36a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:48.503994 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-dc01a608-171b-43e6-b377-f56bff2a36a4 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:48.505268 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-dc01a608-171b-43e6-b377-f56bff2a36a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:48.519125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1056/2127] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:48.523781 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e130b-af2e-4deb-bd17-a2c037f6def0 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:48.533856 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e130b-af2e-4deb-bd17-a2c037f6def0 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:48.539676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e1070a-d2d7-4c8b-91f9-c7ed0ea39c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:48.540319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1070a-d2d7-4c8b-91f9-c7ed0ea39c6a admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:48.540593 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1070a-d2d7-4c8b-91f9-c7ed0ea39c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:48.540860 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1070a-d2d7-4c8b-91f9-c7ed0ea39c6a admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:48.541637 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e1070a-d2d7-4c8b-91f9-c7ed0ea39c6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:13:48.541939 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e1070a-d2d7-4c8b-91f9-c7ed0ea39c6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:48.543569 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e1070a-d2d7-4c8b-91f9-c7ed0ea39c6a admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:48.557108 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1072/2128] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:13:48.562597 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc01a608-171b-43e6-b377-f56bff2a36a4 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:48.580670 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc01a608-171b-43e6-b377-f56bff2a36a4 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:48.593902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ec022-d76e-4ae5-9c22-c71778c1640f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:48.595120 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ec022-d76e-4ae5-9c22-c71778c1640f admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:48.595607 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ec022-d76e-4ae5-9c22-c71778c1640f admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:48.595994 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ec022-d76e-4ae5-9c22-c71778c1640f admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:48.597364 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ec022-d76e-4ae5-9c22-c71778c1640f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:48.601711 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ec022-d76e-4ae5-9c22-c71778c1640f admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:48.631086 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:48.663388 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1057/2129] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:13:48 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:48.664051 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e1070a-d2d7-4c8b-91f9-c7ed0ea39c6a admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:48.679987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e1070a-d2d7-4c8b-91f9-c7ed0ea39c6a admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:48.757243 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1935fe-d43f-4173-947d-e6748f8fd88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:48.757791 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1935fe-d43f-4173-947d-e6748f8fd88f admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:48.758068 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1935fe-d43f-4173-947d-e6748f8fd88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:48.758362 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1935fe-d43f-4173-947d-e6748f8fd88f admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:48.759149 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1935fe-d43f-4173-947d-e6748f8fd88f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:48.761233 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1935fe-d43f-4173-947d-e6748f8fd88f admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:48.795078 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1073/2130] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:13:48 2022] POST /identity/v3/users => generated 294 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:48.797778 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ec022-d76e-4ae5-9c22-c71778c1640f admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:48.810885 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ec022-d76e-4ae5-9c22-c71778c1640f admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:48.821373 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a1a601-0df6-42d1-b2f9-dcd748195440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:48.821373 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a1a601-0df6-42d1-b2f9-dcd748195440 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:48.821373 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a1a601-0df6-42d1-b2f9-dcd748195440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:48.821373 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a1a601-0df6-42d1-b2f9-dcd748195440 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:48.821758 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a1a601-0df6-42d1-b2f9-dcd748195440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:48.821758 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a1a601-0df6-42d1-b2f9-dcd748195440 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:48.832815 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1058/2131] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:48.837347 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1935fe-d43f-4173-947d-e6748f8fd88f admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:48.850994 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1935fe-d43f-4173-947d-e6748f8fd88f admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:48.857405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43612859-4b46-4a68-b69d-f6286a864a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:48.858158 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43612859-4b46-4a68-b69d-f6286a864a6c admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:48.858471 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43612859-4b46-4a68-b69d-f6286a864a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:48.858741 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43612859-4b46-4a68-b69d-f6286a864a6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:48.859522 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43612859-4b46-4a68-b69d-f6286a864a6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:48.861642 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43612859-4b46-4a68-b69d-f6286a864a6c admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:48.881631 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1074/2132] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:48.886101 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a1a601-0df6-42d1-b2f9-dcd748195440 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:48.898008 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a1a601-0df6-42d1-b2f9-dcd748195440 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:48.902980 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a26f3c-b141-471b-a17c-5dc2a36b0f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:48.903975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a26f3c-b141-471b-a17c-5dc2a36b0f32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:48.904322 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a26f3c-b141-471b-a17c-5dc2a36b0f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:48.904580 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a26f3c-b141-471b-a17c-5dc2a36b0f32 admin admin] PATH_INFO: `/v3/projects/5c2c11c738124ef8888c8de03b2822ff/users/b9b739984d074328b69c7c680feb8324/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:48.926686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a26f3c-b141-471b-a17c-5dc2a36b0f32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c2c11c738124ef8888c8de03b2822ff, user_id=b9b739984d074328b69c7c680feb8324, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:48.929944 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a26f3c-b141-471b-a17c-5dc2a36b0f32 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:48.948970 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1059/2133] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:48 2022] PUT /identity/v3/projects/5c2c11c738124ef8888c8de03b2822ff/users/b9b739984d074328b69c7c680feb8324/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:48.968491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e520d-469d-45a8-ae46-822a865f6564 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:48.968772 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e520d-469d-45a8-ae46-822a865f6564 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:48.969043 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e520d-469d-45a8-ae46-822a865f6564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:49.002446 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-a56e520d-469d-45a8-ae46-822a865f6564 None None] MFA Rules not processed for user `b9b739984d074328b69c7c680feb8324`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:49.003875 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-a56e520d-469d-45a8-ae46-822a865f6564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:49.108128 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1075/2134] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:13:49.113523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a26f3c-b141-471b-a17c-5dc2a36b0f32 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:49.207328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a26f3c-b141-471b-a17c-5dc2a36b0f32 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:49.212523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51d47f-612f-40a1-8e05-c101c6e91f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:49.213471 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51d47f-612f-40a1-8e05-c101c6e91f95 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:49.213932 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51d47f-612f-40a1-8e05-c101c6e91f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:49.214379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51d47f-612f-40a1-8e05-c101c6e91f95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:49.215165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba51d47f-612f-40a1-8e05-c101c6e91f95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:49.217673 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba51d47f-612f-40a1-8e05-c101c6e91f95 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:49.257653 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1060/2135] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:13:49 2022] POST /identity/v3/projects => generated 387 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:49.264932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e520d-469d-45a8-ae46-822a865f6564 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:49.286872 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e520d-469d-45a8-ae46-822a865f6564 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:49.329998 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:49.419986 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9682b-91cf-47cb-984c-1a5a648f1da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:49.420813 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9682b-91cf-47cb-984c-1a5a648f1da7 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:49.420962 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9682b-91cf-47cb-984c-1a5a648f1da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:49.421145 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9682b-91cf-47cb-984c-1a5a648f1da7 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:49.429907 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9682b-91cf-47cb-984c-1a5a648f1da7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:49.434983 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9682b-91cf-47cb-984c-1a5a648f1da7 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:49.490833 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1076/2136] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:13:49 2022] POST /identity/v3/users => generated 300 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:49.492837 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51d47f-612f-40a1-8e05-c101c6e91f95 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:49.513481 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51d47f-612f-40a1-8e05-c101c6e91f95 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:49.519671 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c397cf8-a9ce-461c-aedb-f580ab2ecc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:49.520656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c397cf8-a9ce-461c-aedb-f580ab2ecc94 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:49.520801 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c397cf8-a9ce-461c-aedb-f580ab2ecc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:49.521171 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c397cf8-a9ce-461c-aedb-f580ab2ecc94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:49.521892 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c397cf8-a9ce-461c-aedb-f580ab2ecc94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:49.523755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c397cf8-a9ce-461c-aedb-f580ab2ecc94 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:49.542074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1061/2137] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:49.556586 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9682b-91cf-47cb-984c-1a5a648f1da7 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:49.573684 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9682b-91cf-47cb-984c-1a5a648f1da7 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:49.580137 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d508f7-48d1-42dc-9297-ccb700c6e82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:49.580823 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d508f7-48d1-42dc-9297-ccb700c6e82c admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:49.581093 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d508f7-48d1-42dc-9297-ccb700c6e82c admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:49.581400 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d508f7-48d1-42dc-9297-ccb700c6e82c admin admin] PATH_INFO: `/v3/projects/258525169d704fc5b18728d0fee92188/users/2fd97b0322634a5faefe551a89ab78a4/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:49.610883 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d508f7-48d1-42dc-9297-ccb700c6e82c admin admin] RBAC: Authorizing `identity:create_grant(project_id=258525169d704fc5b18728d0fee92188, user_id=2fd97b0322634a5faefe551a89ab78a4, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:49.616492 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d508f7-48d1-42dc-9297-ccb700c6e82c admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:49.642981 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1077/2138] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:49 2022] PUT /identity/v3/projects/258525169d704fc5b18728d0fee92188/users/2fd97b0322634a5faefe551a89ab78a4/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:49.649956 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c397cf8-a9ce-461c-aedb-f580ab2ecc94 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:49.669083 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c397cf8-a9ce-461c-aedb-f580ab2ecc94 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:49.777769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963cad1f-16ba-49c4-8c55-889187a02292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:49.778418 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963cad1f-16ba-49c4-8c55-889187a02292 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:49.778690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963cad1f-16ba-49c4-8c55-889187a02292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:49.778966 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963cad1f-16ba-49c4-8c55-889187a02292 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:49.782794 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963cad1f-16ba-49c4-8c55-889187a02292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:49.782794 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963cad1f-16ba-49c4-8c55-889187a02292 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:49.800284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1062/2139] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:49.800741 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d508f7-48d1-42dc-9297-ccb700c6e82c admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:49.813869 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d508f7-48d1-42dc-9297-ccb700c6e82c admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:49.819209 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535b142-cfd8-4ec6-a641-d217613399fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:49.819897 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535b142-cfd8-4ec6-a641-d217613399fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:49.820186 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535b142-cfd8-4ec6-a641-d217613399fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:49.820602 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535b142-cfd8-4ec6-a641-d217613399fa admin admin] PATH_INFO: `/v3/projects/258525169d704fc5b18728d0fee92188/users/2fd97b0322634a5faefe551a89ab78a4/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:49.822692 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535b142-cfd8-4ec6-a641-d217613399fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=258525169d704fc5b18728d0fee92188, user_id=2fd97b0322634a5faefe551a89ab78a4, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:49.825764 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535b142-cfd8-4ec6-a641-d217613399fa admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:49.842983 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1078/2140] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:49 2022] PUT /identity/v3/projects/258525169d704fc5b18728d0fee92188/users/2fd97b0322634a5faefe551a89ab78a4/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:13:49.852474 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec26bf5-6574-42d5-9dca-fd014a8528e4 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:49.852760 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec26bf5-6574-42d5-9dca-fd014a8528e4 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:49.853090 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec26bf5-6574-42d5-9dca-fd014a8528e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:49.887887 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-5ec26bf5-6574-42d5-9dca-fd014a8528e4 None None] MFA Rules not processed for user `2fd97b0322634a5faefe551a89ab78a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:49.889180 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-5ec26bf5-6574-42d5-9dca-fd014a8528e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:49.916004 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:50.003831 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1063/2141] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:49 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:13:50.015585 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535b142-cfd8-4ec6-a641-d217613399fa admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:50.017552 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-1535b142-cfd8-4ec6-a641-d217613399fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:50.095042 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535b142-cfd8-4ec6-a641-d217613399fa admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:50.100317 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff2e22-f47d-4572-bfe7-80443e6fd921 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:50.100963 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff2e22-f47d-4572-bfe7-80443e6fd921 service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:50.101280 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff2e22-f47d-4572-bfe7-80443e6fd921 service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:50.101554 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff2e22-f47d-4572-bfe7-80443e6fd921 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:50.112970 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff2e22-f47d-4572-bfe7-80443e6fd921 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:50.114869 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff2e22-f47d-4572-bfe7-80443e6fd921 service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:50.126106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1079/2142] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:50 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:50.569180 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec26bf5-6574-42d5-9dca-fd014a8528e4 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:50.571849 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-5ec26bf5-6574-42d5-9dca-fd014a8528e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:50.718352 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec26bf5-6574-42d5-9dca-fd014a8528e4 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:50.729763 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25c1b5-57af-438c-852b-2796862074d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:50.731052 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25c1b5-57af-438c-852b-2796862074d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:50.731395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25c1b5-57af-438c-852b-2796862074d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:50.731887 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25c1b5-57af-438c-852b-2796862074d8 admin admin] PATH_INFO: `/v3/users/a7e0c489e68646afaf64c8cd0f6a10ed` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:50.733989 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25c1b5-57af-438c-852b-2796862074d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7e0c489e68646afaf64c8cd0f6a10ed)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:50.737804 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25c1b5-57af-438c-852b-2796862074d8 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:50.852828 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-2c25c1b5-57af-438c-852b-2796862074d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7e0c489e68646afaf64c8cd0f6a10ed'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:50.869565 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-2c25c1b5-57af-438c-852b-2796862074d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7e0c489e68646afaf64c8cd0f6a10ed'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:50.907390 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-2c25c1b5-57af-438c-852b-2796862074d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7e0c489e68646afaf64c8cd0f6a10ed'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:50.911293 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-2c25c1b5-57af-438c-852b-2796862074d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7e0c489e68646afaf64c8cd0f6a10ed'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:50.937748 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-2c25c1b5-57af-438c-852b-2796862074d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7e0c489e68646afaf64c8cd0f6a10ed'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:50.961441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1064/2143] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:50 2022] DELETE /identity/v3/users/a7e0c489e68646afaf64c8cd0f6a10ed => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:50.967141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53e0c2-46e4-4e41-8033-3bd0c9763bd5 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:50.967909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53e0c2-46e4-4e41-8033-3bd0c9763bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:50.968485 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53e0c2-46e4-4e41-8033-3bd0c9763bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:51.020192 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-3b53e0c2-46e4-4e41-8033-3bd0c9763bd5 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:51.022587 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-3b53e0c2-46e4-4e41-8033-3bd0c9763bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:51.139584 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25c1b5-57af-438c-852b-2796862074d8 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:51.141834 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-2c25c1b5-57af-438c-852b-2796862074d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:51.152005 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1080/2144] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:51.166117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25c1b5-57af-438c-852b-2796862074d8 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:51.168794 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b53e0c2-46e4-4e41-8033-3bd0c9763bd5 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:51.171675 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-3b53e0c2-46e4-4e41-8033-3bd0c9763bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:51.174952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737b12b-2fa8-4515-92ac-d4e7778ec2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:51.176212 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737b12b-2fa8-4515-92ac-d4e7778ec2ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:51.176793 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737b12b-2fa8-4515-92ac-d4e7778ec2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:51.177404 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737b12b-2fa8-4515-92ac-d4e7778ec2ee admin admin] PATH_INFO: `/v3/users/b9b739984d074328b69c7c680feb8324` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:51.179046 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737b12b-2fa8-4515-92ac-d4e7778ec2ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9b739984d074328b69c7c680feb8324)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:51.184528 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737b12b-2fa8-4515-92ac-d4e7778ec2ee admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:51.233902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:51.239756 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:51.304660 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-6737b12b-2fa8-4515-92ac-d4e7778ec2ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9b739984d074328b69c7c680feb8324'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:51.326198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-6737b12b-2fa8-4515-92ac-d4e7778ec2ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9b739984d074328b69c7c680feb8324'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:51.344786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b53e0c2-46e4-4e41-8033-3bd0c9763bd5 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:51.352529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c464cd-349c-4723-9859-f32f54bad205 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:51.353973 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c464cd-349c-4723-9859-f32f54bad205 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:51.354394 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c464cd-349c-4723-9859-f32f54bad205 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:51.354811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c464cd-349c-4723-9859-f32f54bad205 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:51.361625 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-6737b12b-2fa8-4515-92ac-d4e7778ec2ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9b739984d074328b69c7c680feb8324'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:51.363163 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-6737b12b-2fa8-4515-92ac-d4e7778ec2ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9b739984d074328b69c7c680feb8324'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:51.369707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c464cd-349c-4723-9859-f32f54bad205 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:51.371397 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c464cd-349c-4723-9859-f32f54bad205 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:51.381769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-6737b12b-2fa8-4515-92ac-d4e7778ec2ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9b739984d074328b69c7c680feb8324'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:51.386398 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1081/2145] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:51.404347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1065/2146] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:51 2022] DELETE /identity/v3/users/b9b739984d074328b69c7c680feb8324 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:13:51.406449 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8265e7-446b-447f-970d-e841cbd4b57f None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:51.406896 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8265e7-446b-447f-970d-e841cbd4b57f None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:51.407344 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8265e7-446b-447f-970d-e841cbd4b57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:51.456614 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-bc8265e7-446b-447f-970d-e841cbd4b57f None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:51.459627 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-bc8265e7-446b-447f-970d-e841cbd4b57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:51.602540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1082/2147] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:13:51.633684 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737b12b-2fa8-4515-92ac-d4e7778ec2ee admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:51.636193 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-6737b12b-2fa8-4515-92ac-d4e7778ec2ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:51.776474 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737b12b-2fa8-4515-92ac-d4e7778ec2ee admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:51.788086 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef58dd-8b12-412f-957e-f77135102365 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:51.789344 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef58dd-8b12-412f-957e-f77135102365 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:51.789978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef58dd-8b12-412f-957e-f77135102365 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:51.790530 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef58dd-8b12-412f-957e-f77135102365 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:51.823444 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef58dd-8b12-412f-957e-f77135102365 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:51.826518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef58dd-8b12-412f-957e-f77135102365 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:51.849261 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1066/2148] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:51.902025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8265e7-446b-447f-970d-e841cbd4b57f None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:51.903800 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-bc8265e7-446b-447f-970d-e841cbd4b57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:51.918813 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8265e7-446b-447f-970d-e841cbd4b57f None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:51.925187 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9135b-5111-49fd-a93e-aaa3070b0578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:51.926076 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9135b-5111-49fd-a93e-aaa3070b0578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:51.926459 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9135b-5111-49fd-a93e-aaa3070b0578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:51.926869 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9135b-5111-49fd-a93e-aaa3070b0578 admin admin] PATH_INFO: `/v3/projects/3292c61ee25f4a5c892feb07fb8c344e` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:51.928955 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9135b-5111-49fd-a93e-aaa3070b0578 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3292c61ee25f4a5c892feb07fb8c344e)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:51.930698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9135b-5111-49fd-a93e-aaa3070b0578 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:52.032651 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-cec9135b-5111-49fd-a93e-aaa3070b0578 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3292c61ee25f4a5c892feb07fb8c344e', 'request_id': 'req-cec9135b-5111-49fd-a93e-aaa3070b0578'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:52.044124 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-cec9135b-5111-49fd-a93e-aaa3070b0578 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3292c61ee25f4a5c892feb07fb8c344e', 'request_id': 'req-cec9135b-5111-49fd-a93e-aaa3070b0578'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:52.067651 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-cec9135b-5111-49fd-a93e-aaa3070b0578 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3292c61ee25f4a5c892feb07fb8c344e', 'request_id': 'req-cec9135b-5111-49fd-a93e-aaa3070b0578'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:52.101063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-cec9135b-5111-49fd-a93e-aaa3070b0578 admin admin] The token cache is being invalidate because project 3292c61ee25f4a5c892feb07fb8c344e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:52.102815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-cec9135b-5111-49fd-a93e-aaa3070b0578 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:52.104134 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-cec9135b-5111-49fd-a93e-aaa3070b0578 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:52.106074 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1083/2149] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:51 2022] DELETE /identity/v3/projects/3292c61ee25f4a5c892feb07fb8c344e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:13:52.113357 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8553410-3c15-46f4-9b9e-53a4c9c963d4 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:52.113759 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8553410-3c15-46f4-9b9e-53a4c9c963d4 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:52.114244 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8553410-3c15-46f4-9b9e-53a4c9c963d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:52.140248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-c8553410-3c15-46f4-9b9e-53a4c9c963d4 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:52.141676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c8553410-3c15-46f4-9b9e-53a4c9c963d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:52.239867 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1067/2150] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:52.248576 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50721102-56bd-42ad-8a5c-bc3ad8abb9a5 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:52.248866 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50721102-56bd-42ad-8a5c-bc3ad8abb9a5 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:52.249665 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50721102-56bd-42ad-8a5c-bc3ad8abb9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:52.296296 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-50721102-56bd-42ad-8a5c-bc3ad8abb9a5 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:52.299144 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-50721102-56bd-42ad-8a5c-bc3ad8abb9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:52.311776 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1084/2151] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:52.313524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8553410-3c15-46f4-9b9e-53a4c9c963d4 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:52.332484 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8553410-3c15-46f4-9b9e-53a4c9c963d4 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:52.341689 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7135a25-058d-40d9-b437-226f952197cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:52.342553 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7135a25-058d-40d9-b437-226f952197cb admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:52.342833 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7135a25-058d-40d9-b437-226f952197cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:52.343317 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7135a25-058d-40d9-b437-226f952197cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:52.344130 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7135a25-058d-40d9-b437-226f952197cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:13:52.344521 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7135a25-058d-40d9-b437-226f952197cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:52.347254 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7135a25-058d-40d9-b437-226f952197cb admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:52.373653 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1068/2152] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:13:52.380613 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50721102-56bd-42ad-8a5c-bc3ad8abb9a5 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:52.398791 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50721102-56bd-42ad-8a5c-bc3ad8abb9a5 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:52.408173 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56211a86-534b-4718-ac7d-12565fd54984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:52.409699 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56211a86-534b-4718-ac7d-12565fd54984 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:52.410170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56211a86-534b-4718-ac7d-12565fd54984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:52.410640 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56211a86-534b-4718-ac7d-12565fd54984 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:52.412067 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56211a86-534b-4718-ac7d-12565fd54984 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:52.415779 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56211a86-534b-4718-ac7d-12565fd54984 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:52.423849 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7135a25-058d-40d9-b437-226f952197cb admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:52.426824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-e7135a25-058d-40d9-b437-226f952197cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:52.455310 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7135a25-058d-40d9-b437-226f952197cb admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:52.470844 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420fe3f8-a901-45cc-b7e5-1907abf13849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:52.472455 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420fe3f8-a901-45cc-b7e5-1907abf13849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:52.472930 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420fe3f8-a901-45cc-b7e5-1907abf13849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:52.473425 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420fe3f8-a901-45cc-b7e5-1907abf13849 admin admin] PATH_INFO: `/v3/projects/5c2c11c738124ef8888c8de03b2822ff` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:52.475647 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420fe3f8-a901-45cc-b7e5-1907abf13849 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c2c11c738124ef8888c8de03b2822ff)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:52.477214 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1085/2153] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:13:52 2022] POST /identity/v3/projects => generated 369 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:52.483839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56211a86-534b-4718-ac7d-12565fd54984 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:52.485235 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420fe3f8-a901-45cc-b7e5-1907abf13849 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:52.522614 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:52.587359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56211a86-534b-4718-ac7d-12565fd54984 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:52.595502 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:52.601077 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fecd6c5-d107-4574-a3d6-db030eb6ce2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:52.602154 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fecd6c5-d107-4574-a3d6-db030eb6ce2e admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:52.602631 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fecd6c5-d107-4574-a3d6-db030eb6ce2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:52.603139 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fecd6c5-d107-4574-a3d6-db030eb6ce2e admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:52.604304 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fecd6c5-d107-4574-a3d6-db030eb6ce2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:52.608082 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fecd6c5-d107-4574-a3d6-db030eb6ce2e admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:52.633099 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-420fe3f8-a901-45cc-b7e5-1907abf13849 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c2c11c738124ef8888c8de03b2822ff', 'request_id': 'req-420fe3f8-a901-45cc-b7e5-1907abf13849'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:52.650034 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1086/2154] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:13:52 2022] POST /identity/v3/users => generated 285 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:52.654216 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-420fe3f8-a901-45cc-b7e5-1907abf13849 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c2c11c738124ef8888c8de03b2822ff', 'request_id': 'req-420fe3f8-a901-45cc-b7e5-1907abf13849'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:52.655176 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fecd6c5-d107-4574-a3d6-db030eb6ce2e admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:52.667708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-420fe3f8-a901-45cc-b7e5-1907abf13849 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c2c11c738124ef8888c8de03b2822ff', 'request_id': 'req-420fe3f8-a901-45cc-b7e5-1907abf13849'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:52.673931 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fecd6c5-d107-4574-a3d6-db030eb6ce2e admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:52.699794 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-420fe3f8-a901-45cc-b7e5-1907abf13849 admin admin] The token cache is being invalidate because project 5c2c11c738124ef8888c8de03b2822ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:52.700389 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-420fe3f8-a901-45cc-b7e5-1907abf13849 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:52.705914 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-420fe3f8-a901-45cc-b7e5-1907abf13849 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:52.710545 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1069/2155] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:52 2022] DELETE /identity/v3/projects/5c2c11c738124ef8888c8de03b2822ff => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:52.717939 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420fe3f8-a901-45cc-b7e5-1907abf13849 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:52.722999 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-420fe3f8-a901-45cc-b7e5-1907abf13849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:52.801581 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f257d7-cbd9-44eb-8c4e-407bfbcc843c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:52.802999 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f257d7-cbd9-44eb-8c4e-407bfbcc843c admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:52.803602 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f257d7-cbd9-44eb-8c4e-407bfbcc843c admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:52.804154 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f257d7-cbd9-44eb-8c4e-407bfbcc843c admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:52.805647 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f257d7-cbd9-44eb-8c4e-407bfbcc843c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:52.808901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f257d7-cbd9-44eb-8c4e-407bfbcc843c admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:52.835536 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1087/2156] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:13:52.842719 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f257d7-cbd9-44eb-8c4e-407bfbcc843c admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:52.856266 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-15f257d7-cbd9-44eb-8c4e-407bfbcc843c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:52.914075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f257d7-cbd9-44eb-8c4e-407bfbcc843c admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:52.924127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d1cdd-f030-4c32-85e2-32b1f335d60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:52.925270 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d1cdd-f030-4c32-85e2-32b1f335d60f admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:52.925864 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d1cdd-f030-4c32-85e2-32b1f335d60f admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:52.926401 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d1cdd-f030-4c32-85e2-32b1f335d60f admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:52.926853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420fe3f8-a901-45cc-b7e5-1907abf13849 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:52.927729 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d1cdd-f030-4c32-85e2-32b1f335d60f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:52.931334 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d1cdd-f030-4c32-85e2-32b1f335d60f admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:52.939710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29039e-129f-4492-a02a-642edda543ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:52.942153 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29039e-129f-4492-a02a-642edda543ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:52.942677 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29039e-129f-4492-a02a-642edda543ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:52.943151 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29039e-129f-4492-a02a-642edda543ec admin admin] PATH_INFO: `/v3/users/2fd97b0322634a5faefe551a89ab78a4` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:52.946645 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc29039e-129f-4492-a02a-642edda543ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fd97b0322634a5faefe551a89ab78a4)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:52.951529 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc29039e-129f-4492-a02a-642edda543ec admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:52.960392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1088/2157] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:13:52.966967 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d1cdd-f030-4c32-85e2-32b1f335d60f admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:53.003409 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d1cdd-f030-4c32-85e2-32b1f335d60f admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:53.021758 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d320ad-0d22-4d23-be2c-ae1051c4d887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:53.024572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d320ad-0d22-4d23-be2c-ae1051c4d887 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:53.025418 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d320ad-0d22-4d23-be2c-ae1051c4d887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:53.029302 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d320ad-0d22-4d23-be2c-ae1051c4d887 admin admin] PATH_INFO: `/v3/projects/6ce565ef16a4439d8b3aad55a2c30758/users/9d20cc2558a94089a64fe8a3800b96a0/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:53.080407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cc29039e-129f-4492-a02a-642edda543ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fd97b0322634a5faefe551a89ab78a4'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:53.085724 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d320ad-0d22-4d23-be2c-ae1051c4d887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ce565ef16a4439d8b3aad55a2c30758, user_id=9d20cc2558a94089a64fe8a3800b96a0, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:53.101320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d320ad-0d22-4d23-be2c-ae1051c4d887 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:53.101610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cc29039e-129f-4492-a02a-642edda543ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fd97b0322634a5faefe551a89ab78a4'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:53.147451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1089/2158] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:52 2022] PUT /identity/v3/projects/6ce565ef16a4439d8b3aad55a2c30758/users/9d20cc2558a94089a64fe8a3800b96a0/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:53.167339 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cc29039e-129f-4492-a02a-642edda543ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fd97b0322634a5faefe551a89ab78a4'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:53.168137 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cc29039e-129f-4492-a02a-642edda543ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fd97b0322634a5faefe551a89ab78a4'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:53.175225 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac4363-312c-44e1-8d2f-2f4974918a9b None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:53.175548 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac4363-312c-44e1-8d2f-2f4974918a9b None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:53.175931 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac4363-312c-44e1-8d2f-2f4974918a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:53.187253 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cc29039e-129f-4492-a02a-642edda543ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fd97b0322634a5faefe551a89ab78a4'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:53.206753 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1070/2159] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:52 2022] DELETE /identity/v3/users/2fd97b0322634a5faefe551a89ab78a4 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:53.223993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-2dac4363-312c-44e1-8d2f-2f4974918a9b None None] MFA Rules not processed for user `9d20cc2558a94089a64fe8a3800b96a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:53.226376 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-2dac4363-312c-44e1-8d2f-2f4974918a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:53.345441 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:53.362497 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1090/2160] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:13:53.370531 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29039e-129f-4492-a02a-642edda543ec admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:53.372486 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-cc29039e-129f-4492-a02a-642edda543ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:53.452405 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:53.492651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29039e-129f-4492-a02a-642edda543ec admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:53.497692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1de111-a0b2-426c-bb30-4467b0d9dd25 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fe5293447346eaafd16818e5bd12f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:53.498543 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1de111-a0b2-426c-bb30-4467b0d9dd25 service glance] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:53.498922 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1de111-a0b2-426c-bb30-4467b0d9dd25 service glance] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:53.499704 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1de111-a0b2-426c-bb30-4467b0d9dd25 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:53.518148 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1de111-a0b2-426c-bb30-4467b0d9dd25 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:53.520259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1de111-a0b2-426c-bb30-4467b0d9dd25 service glance] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:53.533924 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1071/2161] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:53 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:53.784643 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac4363-312c-44e1-8d2f-2f4974918a9b None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:53.797493 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-2dac4363-312c-44e1-8d2f-2f4974918a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:53.978468 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac4363-312c-44e1-8d2f-2f4974918a9b None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:53.986265 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de86b9-9780-4366-a3d3-350b08d536da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:53.987718 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de86b9-9780-4366-a3d3-350b08d536da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:53.988107 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de86b9-9780-4366-a3d3-350b08d536da admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:53.988356 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de86b9-9780-4366-a3d3-350b08d536da admin admin] PATH_INFO: `/v3/projects/258525169d704fc5b18728d0fee92188` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:53.990222 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de86b9-9780-4366-a3d3-350b08d536da admin admin] RBAC: Authorizing `identity:delete_project(project_id=258525169d704fc5b18728d0fee92188)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:53.995710 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de86b9-9780-4366-a3d3-350b08d536da admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:54.112011 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-35de86b9-9780-4366-a3d3-350b08d536da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '258525169d704fc5b18728d0fee92188', 'request_id': 'req-35de86b9-9780-4366-a3d3-350b08d536da'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:54.130599 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-35de86b9-9780-4366-a3d3-350b08d536da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '258525169d704fc5b18728d0fee92188', 'request_id': 'req-35de86b9-9780-4366-a3d3-350b08d536da'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:54.144379 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-35de86b9-9780-4366-a3d3-350b08d536da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '258525169d704fc5b18728d0fee92188', 'request_id': 'req-35de86b9-9780-4366-a3d3-350b08d536da'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:54.176524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-35de86b9-9780-4366-a3d3-350b08d536da admin admin] The token cache is being invalidate because project 258525169d704fc5b18728d0fee92188 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:54.177552 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-35de86b9-9780-4366-a3d3-350b08d536da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:54.178718 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-35de86b9-9780-4366-a3d3-350b08d536da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:54.184777 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1091/2162] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:53 2022] DELETE /identity/v3/projects/258525169d704fc5b18728d0fee92188 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:54.187137 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9f126-fabc-4ba1-95d7-0ce1bac565f1 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:54.188358 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9f126-fabc-4ba1-95d7-0ce1bac565f1 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:54.188661 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9f126-fabc-4ba1-95d7-0ce1bac565f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:54.232780 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-b4c9f126-fabc-4ba1-95d7-0ce1bac565f1 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:54.234400 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b4c9f126-fabc-4ba1-95d7-0ce1bac565f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:54.381829 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1072/2163] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:54.382331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeaddba-9d53-4a56-b3a5-c0731ceb2f75 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:54.382688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeaddba-9d53-4a56-b3a5-c0731ceb2f75 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:54.384744 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeaddba-9d53-4a56-b3a5-c0731ceb2f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:54.421904 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-1aeaddba-9d53-4a56-b3a5-c0731ceb2f75 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:54.423146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-1aeaddba-9d53-4a56-b3a5-c0731ceb2f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:54.436298 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1092/2164] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:13:54.437765 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9f126-fabc-4ba1-95d7-0ce1bac565f1 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:54.450960 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9f126-fabc-4ba1-95d7-0ce1bac565f1 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:54.458893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c442e9a-ebc9-4521-a15b-993169d10b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:54.460490 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c442e9a-ebc9-4521-a15b-993169d10b1d admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:54.460490 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c442e9a-ebc9-4521-a15b-993169d10b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:54.460490 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c442e9a-ebc9-4521-a15b-993169d10b1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:54.461077 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c442e9a-ebc9-4521-a15b-993169d10b1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:13:54.461481 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c442e9a-ebc9-4521-a15b-993169d10b1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:54.463446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c442e9a-ebc9-4521-a15b-993169d10b1d admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:54.478420 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1073/2165] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:13:54.483531 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeaddba-9d53-4a56-b3a5-c0731ceb2f75 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:54.496471 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeaddba-9d53-4a56-b3a5-c0731ceb2f75 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:54.502720 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9775b571-817c-4b4b-9235-30c227eab8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:54.503748 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775b571-817c-4b4b-9235-30c227eab8a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:54.504161 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775b571-817c-4b4b-9235-30c227eab8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:54.504588 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775b571-817c-4b4b-9235-30c227eab8a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:54.505869 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9775b571-817c-4b4b-9235-30c227eab8a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:54.508699 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9775b571-817c-4b4b-9235-30c227eab8a7 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:54.562771 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1093/2166] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:13:54 2022] POST /identity/v3/projects => generated 385 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:54.567377 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c442e9a-ebc9-4521-a15b-993169d10b1d admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:54.579410 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c442e9a-ebc9-4521-a15b-993169d10b1d admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:54.712484 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23fb93-9511-4b5a-ab97-f60b5e7a37f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:54.713161 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23fb93-9511-4b5a-ab97-f60b5e7a37f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:54.713443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23fb93-9511-4b5a-ab97-f60b5e7a37f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:54.716212 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23fb93-9511-4b5a-ab97-f60b5e7a37f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:54.716974 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb23fb93-9511-4b5a-ab97-f60b5e7a37f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:54.719112 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb23fb93-9511-4b5a-ab97-f60b5e7a37f3 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:54.780736 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1074/2167] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:13:54 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:54.789115 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9775b571-817c-4b4b-9235-30c227eab8a7 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:54.808622 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9775b571-817c-4b4b-9235-30c227eab8a7 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:54.815250 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb7b4a4-d560-46dd-84cb-f8bc99ee02e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:54.816113 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7b4a4-d560-46dd-84cb-f8bc99ee02e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:54.816286 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7b4a4-d560-46dd-84cb-f8bc99ee02e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:54.816547 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7b4a4-d560-46dd-84cb-f8bc99ee02e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:54.817427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb7b4a4-d560-46dd-84cb-f8bc99ee02e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:54.820335 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb7b4a4-d560-46dd-84cb-f8bc99ee02e6 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:54.845080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1094/2168] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:13:54.850296 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23fb93-9511-4b5a-ab97-f60b5e7a37f3 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:54.911935 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23fb93-9511-4b5a-ab97-f60b5e7a37f3 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:54.928452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97dc58-32d6-4f05-9c20-13f3da9bea22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:54.929267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97dc58-32d6-4f05-9c20-13f3da9bea22 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:54.929593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97dc58-32d6-4f05-9c20-13f3da9bea22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:54.929880 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97dc58-32d6-4f05-9c20-13f3da9bea22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:54.930583 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e97dc58-32d6-4f05-9c20-13f3da9bea22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:54.938113 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e97dc58-32d6-4f05-9c20-13f3da9bea22 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:54.956236 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1075/2169] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:54.961363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb7b4a4-d560-46dd-84cb-f8bc99ee02e6 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:54.975351 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb7b4a4-d560-46dd-84cb-f8bc99ee02e6 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:54.983085 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6fa9be-28f1-4eeb-8859-d17df1d732d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:54.983879 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6fa9be-28f1-4eeb-8859-d17df1d732d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:54.984161 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6fa9be-28f1-4eeb-8859-d17df1d732d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:54.984662 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6fa9be-28f1-4eeb-8859-d17df1d732d6 admin admin] PATH_INFO: `/v3/projects/4f332abb00db4704a832fab5f85615b6/users/ed98ed05742741b78c11f3483c786588/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:55.012061 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6fa9be-28f1-4eeb-8859-d17df1d732d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f332abb00db4704a832fab5f85615b6, user_id=ed98ed05742741b78c11f3483c786588, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:55.015088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6fa9be-28f1-4eeb-8859-d17df1d732d6 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:55.034883 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1095/2170] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:54 2022] PUT /identity/v3/projects/4f332abb00db4704a832fab5f85615b6/users/ed98ed05742741b78c11f3483c786588/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:13:55.048432 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb9fc9-0a0b-45c5-b11e-98d20997b27f None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:55.048717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb9fc9-0a0b-45c5-b11e-98d20997b27f None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:55.048993 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb9fc9-0a0b-45c5-b11e-98d20997b27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:55.094156 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-64cb9fc9-0a0b-45c5-b11e-98d20997b27f None None] MFA Rules not processed for user `ed98ed05742741b78c11f3483c786588`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:55.096120 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-64cb9fc9-0a0b-45c5-b11e-98d20997b27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:55.146784 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:55.211586 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1076/2171] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:55.217205 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6fa9be-28f1-4eeb-8859-d17df1d732d6 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:55.297115 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:55.372896 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6fa9be-28f1-4eeb-8859-d17df1d732d6 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:55.400070 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75763105-1e14-4d2c-b432-c542864544cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:55.400767 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75763105-1e14-4d2c-b432-c542864544cd admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:55.401229 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75763105-1e14-4d2c-b432-c542864544cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:55.401651 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75763105-1e14-4d2c-b432-c542864544cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:55.402483 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75763105-1e14-4d2c-b432-c542864544cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:55.404603 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75763105-1e14-4d2c-b432-c542864544cd admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:55.474902 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1096/2172] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:13:55 2022] POST /identity/v3/projects => generated 383 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:55.480243 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb9fc9-0a0b-45c5-b11e-98d20997b27f None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:55.521713 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb9fc9-0a0b-45c5-b11e-98d20997b27f None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:55.613778 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b720d-06d1-47d9-bdc7-7d6dabb76e1f None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:55.614051 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b720d-06d1-47d9-bdc7-7d6dabb76e1f None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:55.614336 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b720d-06d1-47d9-bdc7-7d6dabb76e1f None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:55.616187 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1097/2173] 10.222.0.44 () {54 vars in 921 bytes} [Sun Sep 11 15:13:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 15:13:55.628429 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be28ba-ab40-4e43-b9b9-04d2fb87c9d0 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:55.628820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be28ba-ab40-4e43-b9b9-04d2fb87c9d0 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:55.629103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be28ba-ab40-4e43-b9b9-04d2fb87c9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:55.644880 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c4205-9925-4dfb-8986-9ea7866dd739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:55.646479 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c4205-9925-4dfb-8986-9ea7866dd739 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:55.646750 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c4205-9925-4dfb-8986-9ea7866dd739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:55.647174 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c4205-9925-4dfb-8986-9ea7866dd739 admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:55.648193 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c4205-9925-4dfb-8986-9ea7866dd739 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:55.650210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c4205-9925-4dfb-8986-9ea7866dd739 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:55.651892 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-f9be28ba-ab40-4e43-b9b9-04d2fb87c9d0 None None] MFA Rules not processed for user `ead06362f8a2444bbfe1405cd602f1c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:55.653144 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-f9be28ba-ab40-4e43-b9b9-04d2fb87c9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:55.679562 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1077/2174] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:13:55 2022] POST /identity/v3/users => generated 298 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:55.684094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c4205-9925-4dfb-8986-9ea7866dd739 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:55.699962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c4205-9925-4dfb-8986-9ea7866dd739 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:55.706987 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd79b0a-0126-409d-8ee2-2f7992713a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:55.707218 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd79b0a-0126-409d-8ee2-2f7992713a03 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:55.707504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd79b0a-0126-409d-8ee2-2f7992713a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:55.717518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd79b0a-0126-409d-8ee2-2f7992713a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:55.718758 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd79b0a-0126-409d-8ee2-2f7992713a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:55.720455 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd79b0a-0126-409d-8ee2-2f7992713a03 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:55.735727 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1078/2175] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:55.736730 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd79b0a-0126-409d-8ee2-2f7992713a03 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:55.746196 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-9fd79b0a-0126-409d-8ee2-2f7992713a03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:55.748540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1098/2176] 10.222.0.44 () {58 vars in 1018 bytes} [Sun Sep 11 15:13:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:13:55.750101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be28ba-ab40-4e43-b9b9-04d2fb87c9d0 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:55.764997 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd79b0a-0126-409d-8ee2-2f7992713a03 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:55.765842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be28ba-ab40-4e43-b9b9-04d2fb87c9d0 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:55.771036 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692f4ec-1b51-4e1b-adc6-4a8f560e85a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:55.771706 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692f4ec-1b51-4e1b-adc6-4a8f560e85a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:55.771987 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692f4ec-1b51-4e1b-adc6-4a8f560e85a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:55.772239 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba830eab-8e82-4320-9242-3077031863ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:55.772475 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692f4ec-1b51-4e1b-adc6-4a8f560e85a9 admin admin] PATH_INFO: `/v3/users/3eb721f100f24930bafd056d29048561` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:55.773171 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba830eab-8e82-4320-9242-3077031863ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:55.773437 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba830eab-8e82-4320-9242-3077031863ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:55.773542 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5692f4ec-1b51-4e1b-adc6-4a8f560e85a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3eb721f100f24930bafd056d29048561)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:55.773699 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba830eab-8e82-4320-9242-3077031863ee admin admin] PATH_INFO: `/v3/projects/4158922c8df6471795b40544c774dccf/users/7b69d3c954394722a4d5174ebe079c2d/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:55.775472 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5692f4ec-1b51-4e1b-adc6-4a8f560e85a9 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:55.795659 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba830eab-8e82-4320-9242-3077031863ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=4158922c8df6471795b40544c774dccf, user_id=7b69d3c954394722a4d5174ebe079c2d, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:55.800175 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba830eab-8e82-4320-9242-3077031863ee admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:55.817098 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1099/2177] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:55 2022] PUT /identity/v3/projects/4158922c8df6471795b40544c774dccf/users/7b69d3c954394722a4d5174ebe079c2d/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:55.818743 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a348a06-3c20-4236-8afa-9e784ca5c1f5 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:55.819065 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a348a06-3c20-4236-8afa-9e784ca5c1f5 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:55.819445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a348a06-3c20-4236-8afa-9e784ca5c1f5 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:55.821189 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1100/2178] 10.222.0.44 () {54 vars in 961 bytes} [Sun Sep 11 15:13:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:13:55.822705 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a348a06-3c20-4236-8afa-9e784ca5c1f5 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:55.844184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-5692f4ec-1b51-4e1b-adc6-4a8f560e85a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3eb721f100f24930bafd056d29048561'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:55.857115 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-5692f4ec-1b51-4e1b-adc6-4a8f560e85a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3eb721f100f24930bafd056d29048561'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:55.880899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-5692f4ec-1b51-4e1b-adc6-4a8f560e85a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3eb721f100f24930bafd056d29048561'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:55.881606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-5692f4ec-1b51-4e1b-adc6-4a8f560e85a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3eb721f100f24930bafd056d29048561'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:55.890919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-5692f4ec-1b51-4e1b-adc6-4a8f560e85a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3eb721f100f24930bafd056d29048561'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:55.900259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1079/2179] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:13:55 2022] DELETE /identity/v3/users/3eb721f100f24930bafd056d29048561 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:55.901754 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6262b3-9e35-48d1-9197-46c511a10757 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:55.902046 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6262b3-9e35-48d1-9197-46c511a10757 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:55.902326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6262b3-9e35-48d1-9197-46c511a10757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:55.940561 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-bd6262b3-9e35-48d1-9197-46c511a10757 None None] MFA Rules not processed for user `ead06362f8a2444bbfe1405cd602f1c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:55.941580 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a348a06-3c20-4236-8afa-9e784ca5c1f5 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:55.942942 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-bd6262b3-9e35-48d1-9197-46c511a10757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:55.990036 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:55.996325 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:56.065213 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1080/2180] 10.222.0.44 () {58 vars in 1058 bytes} [Sun Sep 11 15:13:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:56.070570 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67475540-43b1-4f32-b506-f3574a23c9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:56.071615 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67475540-43b1-4f32-b506-f3574a23c9a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:56.072169 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67475540-43b1-4f32-b506-f3574a23c9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:56.072666 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67475540-43b1-4f32-b506-f3574a23c9a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:56.076127 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6262b3-9e35-48d1-9197-46c511a10757 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:56.082074 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67475540-43b1-4f32-b506-f3574a23c9a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:56.086081 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67475540-43b1-4f32-b506-f3574a23c9a1 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:56.100907 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6262b3-9e35-48d1-9197-46c511a10757 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:56.110891 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1101/2181] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:13:56.116029 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67475540-43b1-4f32-b506-f3574a23c9a1 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:56.119219 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-67475540-43b1-4f32-b506-f3574a23c9a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:56.124974 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593653be-346c-4ec9-8dc7-2982501559ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead06362f8a2444bbfe1405cd602f1c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:56.126725 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593653be-346c-4ec9-8dc7-2982501559ae service placement] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:56.127058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593653be-346c-4ec9-8dc7-2982501559ae service placement] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:56.127777 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593653be-346c-4ec9-8dc7-2982501559ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:56.131910 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-593653be-346c-4ec9-8dc7-2982501559ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:56.145297 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67475540-43b1-4f32-b506-f3574a23c9a1 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:56.153496 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45428ceb-4d20-4bd1-b5b5-4b8404b32ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:56.154628 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45428ceb-4d20-4bd1-b5b5-4b8404b32ba7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:56.155121 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45428ceb-4d20-4bd1-b5b5-4b8404b32ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:56.155601 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45428ceb-4d20-4bd1-b5b5-4b8404b32ba7 admin admin] PATH_INFO: `/v3/projects/4158922c8df6471795b40544c774dccf/users/7b69d3c954394722a4d5174ebe079c2d/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:56.161113 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45428ceb-4d20-4bd1-b5b5-4b8404b32ba7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4158922c8df6471795b40544c774dccf, user_id=7b69d3c954394722a4d5174ebe079c2d, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:56.163808 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593653be-346c-4ec9-8dc7-2982501559ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:56.166130 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593653be-346c-4ec9-8dc7-2982501559ae service placement] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:56.167116 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45428ceb-4d20-4bd1-b5b5-4b8404b32ba7 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:56.182737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1081/2182] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:13:56.184576 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593653be-346c-4ec9-8dc7-2982501559ae service placement] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:56.188710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-593653be-346c-4ec9-8dc7-2982501559ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:56.200978 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1102/2183] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:56 2022] PUT /identity/v3/projects/4158922c8df6471795b40544c774dccf/users/7b69d3c954394722a4d5174ebe079c2d/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:56.213930 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593653be-346c-4ec9-8dc7-2982501559ae service placement] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:56.220091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7feaf4-fc4c-449c-a0a7-0dc187e3c94e None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:56.220837 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7feaf4-fc4c-449c-a0a7-0dc187e3c94e None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:56.221380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7feaf4-fc4c-449c-a0a7-0dc187e3c94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:56.271150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-dc7feaf4-fc4c-449c-a0a7-0dc187e3c94e None None] MFA Rules not processed for user `7b69d3c954394722a4d5174ebe079c2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:56.273602 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-dc7feaf4-fc4c-449c-a0a7-0dc187e3c94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:56.297960 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e2271-533b-4b02-a2ed-c8de358d3d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:56.298649 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e2271-533b-4b02-a2ed-c8de358d3d43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:56.298928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e2271-533b-4b02-a2ed-c8de358d3d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:56.299237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e2271-533b-4b02-a2ed-c8de358d3d43 admin admin] PATH_INFO: `/v3/projects/a9e42cc8e82e41e2a5a33ae922079bc8` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:56.300563 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936e2271-533b-4b02-a2ed-c8de358d3d43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9e42cc8e82e41e2a5a33ae922079bc8)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:56.302662 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936e2271-533b-4b02-a2ed-c8de358d3d43 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:56.364092 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-936e2271-533b-4b02-a2ed-c8de358d3d43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9e42cc8e82e41e2a5a33ae922079bc8', 'request_id': 'req-936e2271-533b-4b02-a2ed-c8de358d3d43'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:56.376713 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-936e2271-533b-4b02-a2ed-c8de358d3d43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9e42cc8e82e41e2a5a33ae922079bc8', 'request_id': 'req-936e2271-533b-4b02-a2ed-c8de358d3d43'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:56.385269 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-936e2271-533b-4b02-a2ed-c8de358d3d43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9e42cc8e82e41e2a5a33ae922079bc8', 'request_id': 'req-936e2271-533b-4b02-a2ed-c8de358d3d43'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:56.403720 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-936e2271-533b-4b02-a2ed-c8de358d3d43 admin admin] The token cache is being invalidate because project a9e42cc8e82e41e2a5a33ae922079bc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:13:56.404125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-936e2271-533b-4b02-a2ed-c8de358d3d43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:56.404960 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-936e2271-533b-4b02-a2ed-c8de358d3d43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:13:56.406636 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1082/2184] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:13:56 2022] DELETE /identity/v3/projects/a9e42cc8e82e41e2a5a33ae922079bc8 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:56.409239 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1103/2185] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:56.415419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e21a7-0375-44d9-bacb-4a8da6ae399e None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:56.415909 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e21a7-0375-44d9-bacb-4a8da6ae399e None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:56.416441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e21a7-0375-44d9-bacb-4a8da6ae399e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:56.425513 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7feaf4-fc4c-449c-a0a7-0dc187e3c94e None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:56.427145 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-dc7feaf4-fc4c-449c-a0a7-0dc187e3c94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:56.447203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-ec3e21a7-0375-44d9-bacb-4a8da6ae399e None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:56.449342 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-ec3e21a7-0375-44d9-bacb-4a8da6ae399e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:56.511189 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:56.525343 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7feaf4-fc4c-449c-a0a7-0dc187e3c94e None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:56.569468 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1083/2186] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:13:56.579391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920102e0-7262-4f7f-aa07-e657df20309b None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:56.579704 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920102e0-7262-4f7f-aa07-e657df20309b None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:56.579930 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920102e0-7262-4f7f-aa07-e657df20309b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:56.606420 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-920102e0-7262-4f7f-aa07-e657df20309b None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:56.607689 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-920102e0-7262-4f7f-aa07-e657df20309b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:56.617050 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1084/2187] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:13:56.622487 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920102e0-7262-4f7f-aa07-e657df20309b None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:56.628556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890f4c2-e3ec-4bcb-a87b-f88d15680df8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:56.629754 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890f4c2-e3ec-4bcb-a87b-f88d15680df8 service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:56.630313 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890f4c2-e3ec-4bcb-a87b-f88d15680df8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:56.632726 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920102e0-7262-4f7f-aa07-e657df20309b None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:56.633814 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890f4c2-e3ec-4bcb-a87b-f88d15680df8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:56.638552 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-5890f4c2-e3ec-4bcb-a87b-f88d15680df8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:56.639907 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f683f-6469-4802-ab2d-1a481f7e80c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:56.640543 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f683f-6469-4802-ab2d-1a481f7e80c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:56.640819 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f683f-6469-4802-ab2d-1a481f7e80c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:56.641090 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f683f-6469-4802-ab2d-1a481f7e80c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:56.641799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f683f-6469-4802-ab2d-1a481f7e80c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:13:56.642098 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f683f-6469-4802-ab2d-1a481f7e80c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:56.643773 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f683f-6469-4802-ab2d-1a481f7e80c3 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:56.649940 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:56.661201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1085/2188] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:13:56.665993 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f683f-6469-4802-ab2d-1a481f7e80c3 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:56.677609 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f683f-6469-4802-ab2d-1a481f7e80c3 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:56.683248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc3179-693d-4db4-adaf-405a3efdd638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:56.684644 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc3179-693d-4db4-adaf-405a3efdd638 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:56.685446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc3179-693d-4db4-adaf-405a3efdd638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:56.685446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc3179-693d-4db4-adaf-405a3efdd638 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:56.686279 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bc3179-693d-4db4-adaf-405a3efdd638 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:56.693456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bc3179-693d-4db4-adaf-405a3efdd638 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:56.736365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1086/2189] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:13:56 2022] POST /identity/v3/projects => generated 357 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:13:56.736698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc3179-693d-4db4-adaf-405a3efdd638 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:56.793529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890f4c2-e3ec-4bcb-a87b-f88d15680df8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:56.796796 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890f4c2-e3ec-4bcb-a87b-f88d15680df8 service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:56.824851 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1104/2190] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:13:56.856267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc3179-693d-4db4-adaf-405a3efdd638 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:56.862489 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e019559-f2a5-4d33-bc4b-287f9f0f7abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:56.863130 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e019559-f2a5-4d33-bc4b-287f9f0f7abc admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:56.863409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e019559-f2a5-4d33-bc4b-287f9f0f7abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:56.863718 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e019559-f2a5-4d33-bc4b-287f9f0f7abc admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:56.864337 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e019559-f2a5-4d33-bc4b-287f9f0f7abc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:56.867684 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e019559-f2a5-4d33-bc4b-287f9f0f7abc admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:56.920467 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1087/2191] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:13:56 2022] POST /identity/v3/users => generated 279 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:13:56.922417 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890f4c2-e3ec-4bcb-a87b-f88d15680df8 service cinder] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:56.948979 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890f4c2-e3ec-4bcb-a87b-f88d15680df8 service cinder] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:56.960225 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd478c4-b765-4342-9a36-f32e00741430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:56.961519 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd478c4-b765-4342-9a36-f32e00741430 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:56.962107 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd478c4-b765-4342-9a36-f32e00741430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:56.962657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd478c4-b765-4342-9a36-f32e00741430 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:56.964029 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd478c4-b765-4342-9a36-f32e00741430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:56.967880 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd478c4-b765-4342-9a36-f32e00741430 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:56.987353 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1105/2192] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:13:56.989618 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e019559-f2a5-4d33-bc4b-287f9f0f7abc admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:57.006601 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e019559-f2a5-4d33-bc4b-287f9f0f7abc admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:57.014855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8db322-c31c-444d-ab19-0f33027cc5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:57.015881 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8db322-c31c-444d-ab19-0f33027cc5fc admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:57.016292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8db322-c31c-444d-ab19-0f33027cc5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:57.016731 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8db322-c31c-444d-ab19-0f33027cc5fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:57.018236 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8db322-c31c-444d-ab19-0f33027cc5fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:57.021273 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8db322-c31c-444d-ab19-0f33027cc5fc admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:57.043177 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1088/2193] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:57.048457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd478c4-b765-4342-9a36-f32e00741430 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:57.061977 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd478c4-b765-4342-9a36-f32e00741430 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:57.068810 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03063499-7cea-4120-8c84-f851979f2972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:57.069534 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03063499-7cea-4120-8c84-f851979f2972 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:57.069816 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03063499-7cea-4120-8c84-f851979f2972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:57.070081 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03063499-7cea-4120-8c84-f851979f2972 admin admin] PATH_INFO: `/v3/projects/5afb78ce354d4279b9926e1f7718846a/users/8d1150f59b7448d7af330cfbb70d32f8/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:57.090129 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03063499-7cea-4120-8c84-f851979f2972 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5afb78ce354d4279b9926e1f7718846a, user_id=8d1150f59b7448d7af330cfbb70d32f8, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:57.093491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03063499-7cea-4120-8c84-f851979f2972 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:57.112230 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1106/2194] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:57 2022] PUT /identity/v3/projects/5afb78ce354d4279b9926e1f7718846a/users/8d1150f59b7448d7af330cfbb70d32f8/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:57.124054 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8b7d2-d854-4e68-8575-b77086f2316b None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:57.124651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8b7d2-d854-4e68-8575-b77086f2316b None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:57.125253 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8b7d2-d854-4e68-8575-b77086f2316b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:57.175200 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-82a8b7d2-d854-4e68-8575-b77086f2316b None None] MFA Rules not processed for user `8d1150f59b7448d7af330cfbb70d32f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:57.177347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-82a8b7d2-d854-4e68-8575-b77086f2316b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:57.289397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1089/2195] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:57 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:57.295408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03063499-7cea-4120-8c84-f851979f2972 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:57.402621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03063499-7cea-4120-8c84-f851979f2972 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:57.416399 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceda86c-bafa-418e-b0a2-67f3e0c601b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:57.417021 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceda86c-bafa-418e-b0a2-67f3e0c601b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:57.417345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceda86c-bafa-418e-b0a2-67f3e0c601b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:57.417616 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceda86c-bafa-418e-b0a2-67f3e0c601b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:57.418363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceda86c-bafa-418e-b0a2-67f3e0c601b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:57.422021 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceda86c-bafa-418e-b0a2-67f3e0c601b0 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:57.467457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1107/2196] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:13:57 2022] POST /identity/v3/projects => generated 357 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:57.473069 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8b7d2-d854-4e68-8575-b77086f2316b None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:57.496696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8b7d2-d854-4e68-8575-b77086f2316b None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:57.503769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:57.616256 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb88d6-a590-460c-a83a-523ab9e7ce03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:57.617367 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb88d6-a590-460c-a83a-523ab9e7ce03 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:57.618183 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb88d6-a590-460c-a83a-523ab9e7ce03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:57.618639 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb88d6-a590-460c-a83a-523ab9e7ce03 admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:57.619836 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb88d6-a590-460c-a83a-523ab9e7ce03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:57.623569 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb88d6-a590-460c-a83a-523ab9e7ce03 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:57.669351 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1090/2197] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:13:57 2022] POST /identity/v3/users => generated 285 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:13:57.671402 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceda86c-bafa-418e-b0a2-67f3e0c601b0 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:57.685981 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceda86c-bafa-418e-b0a2-67f3e0c601b0 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:57.692309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423485aa-ab7d-4c80-8e7c-8c2b411d6901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:57.693633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423485aa-ab7d-4c80-8e7c-8c2b411d6901 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:57.693985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423485aa-ab7d-4c80-8e7c-8c2b411d6901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:57.694287 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423485aa-ab7d-4c80-8e7c-8c2b411d6901 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:57.697045 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423485aa-ab7d-4c80-8e7c-8c2b411d6901 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:57.701012 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423485aa-ab7d-4c80-8e7c-8c2b411d6901 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:57.715868 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1108/2198] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:13:57.722356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb88d6-a590-460c-a83a-523ab9e7ce03 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:57.735010 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb88d6-a590-460c-a83a-523ab9e7ce03 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:57.743402 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe5147-40e9-47f5-847a-1b4ba72d8b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:57.744501 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe5147-40e9-47f5-847a-1b4ba72d8b66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:57.744986 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe5147-40e9-47f5-847a-1b4ba72d8b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:57.745477 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe5147-40e9-47f5-847a-1b4ba72d8b66 admin admin] PATH_INFO: `/v3/projects/4e72d2e0e625441986ed00874e3af0ea/users/cdc9506a31344f98a914b91612d1cb72/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:57.770385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe5147-40e9-47f5-847a-1b4ba72d8b66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e72d2e0e625441986ed00874e3af0ea, user_id=cdc9506a31344f98a914b91612d1cb72, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:57.773284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe5147-40e9-47f5-847a-1b4ba72d8b66 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:57.793286 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1091/2199] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:57 2022] PUT /identity/v3/projects/4e72d2e0e625441986ed00874e3af0ea/users/cdc9506a31344f98a914b91612d1cb72/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:57.798462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423485aa-ab7d-4c80-8e7c-8c2b411d6901 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:57.809580 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423485aa-ab7d-4c80-8e7c-8c2b411d6901 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:57.864626 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:13:57.885098 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f9b39-ef00-4c2a-99aa-1767bb21b37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:57.885901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f9b39-ef00-4c2a-99aa-1767bb21b37f admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:57.886162 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f9b39-ef00-4c2a-99aa-1767bb21b37f admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:57.886429 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f9b39-ef00-4c2a-99aa-1767bb21b37f admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:57.887096 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f9b39-ef00-4c2a-99aa-1767bb21b37f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:57.888837 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f9b39-ef00-4c2a-99aa-1767bb21b37f admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:57.900804 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1109/2200] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:13:57.905921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe5147-40e9-47f5-847a-1b4ba72d8b66 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:57.919225 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe5147-40e9-47f5-847a-1b4ba72d8b66 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:57.925231 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaba9b4-d70c-4265-8df0-570f755e468d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:57.925959 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaba9b4-d70c-4265-8df0-570f755e468d admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:57.926213 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaba9b4-d70c-4265-8df0-570f755e468d admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:57.926481 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaba9b4-d70c-4265-8df0-570f755e468d admin admin] PATH_INFO: `/v3/projects/4e72d2e0e625441986ed00874e3af0ea/users/cdc9506a31344f98a914b91612d1cb72/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:57.929181 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaba9b4-d70c-4265-8df0-570f755e468d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e72d2e0e625441986ed00874e3af0ea, user_id=cdc9506a31344f98a914b91612d1cb72, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:57.931939 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaba9b4-d70c-4265-8df0-570f755e468d admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:57.947147 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1092/2201] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:13:57 2022] PUT /identity/v3/projects/4e72d2e0e625441986ed00874e3af0ea/users/cdc9506a31344f98a914b91612d1cb72/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:13:57.962159 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c7026-9b97-4d70-a411-0af5d61b652c None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:57.962608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c7026-9b97-4d70-a411-0af5d61b652c None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:57.963042 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c7026-9b97-4d70-a411-0af5d61b652c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:58.007921 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-9c2c7026-9b97-4d70-a411-0af5d61b652c None None] MFA Rules not processed for user `cdc9506a31344f98a914b91612d1cb72`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:13:58.009988 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-9c2c7026-9b97-4d70-a411-0af5d61b652c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:58.129819 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1110/2202] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:13:57 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:13:58.145032 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaba9b4-d70c-4265-8df0-570f755e468d admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:58.147969 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-aeaba9b4-d70c-4265-8df0-570f755e468d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:13:58.239908 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaba9b4-d70c-4265-8df0-570f755e468d admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:58.244698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504125a-de31-41ae-9e24-ed7ce9b9eb4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:58.245588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504125a-de31-41ae-9e24-ed7ce9b9eb4b service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:58.245859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504125a-de31-41ae-9e24-ed7ce9b9eb4b service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:58.246127 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504125a-de31-41ae-9e24-ed7ce9b9eb4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:58.266112 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e504125a-de31-41ae-9e24-ed7ce9b9eb4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:58.267694 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e504125a-de31-41ae-9e24-ed7ce9b9eb4b service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:58.285309 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1093/2203] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:58 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:13:58.393305 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c7026-9b97-4d70-a411-0af5d61b652c None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:13:58.404283 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c7026-9b97-4d70-a411-0af5d61b652c None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:13:58.408197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f4d02-6ae7-424a-89fb-c9e1dfd3fc7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:13:58.408819 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f4d02-6ae7-424a-89fb-c9e1dfd3fc7d service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:13:58.409093 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f4d02-6ae7-424a-89fb-c9e1dfd3fc7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:13:58.409395 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f4d02-6ae7-424a-89fb-c9e1dfd3fc7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:13:58.481950 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f4d02-6ae7-424a-89fb-c9e1dfd3fc7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:13:58.484499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f4d02-6ae7-424a-89fb-c9e1dfd3fc7d service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:13:58.499373 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1111/2204] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:13:58 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:14:01.861162 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504125a-de31-41ae-9e24-ed7ce9b9eb4b service cinder] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:01.862741 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-e504125a-de31-41ae-9e24-ed7ce9b9eb4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:01.876653 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504125a-de31-41ae-9e24-ed7ce9b9eb4b service cinder] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:01.880639 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213d65c-cf8c-49b4-85a4-60b32e89ba25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:01.881316 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213d65c-cf8c-49b4-85a4-60b32e89ba25 service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:01.881590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213d65c-cf8c-49b4-85a4-60b32e89ba25 service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:01.881860 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213d65c-cf8c-49b4-85a4-60b32e89ba25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:01.883730 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-2213d65c-cf8c-49b4-85a4-60b32e89ba25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:01.991318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:14:01.994845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213d65c-cf8c-49b4-85a4-60b32e89ba25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:01.997586 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213d65c-cf8c-49b4-85a4-60b32e89ba25 service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:02.020043 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1094/2205] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:14:01 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:14:08.492812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=70 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:14:09.980634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f4d02-6ae7-424a-89fb-c9e1dfd3fc7d service cinder] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:09.983031 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-297f4d02-6ae7-424a-89fb-c9e1dfd3fc7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:10.142528 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f4d02-6ae7-424a-89fb-c9e1dfd3fc7d service cinder] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:10.159382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf9e54-1b65-4f84-a48b-ea70a5dbf3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:10.159382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf9e54-1b65-4f84-a48b-ea70a5dbf3f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:10.159382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf9e54-1b65-4f84-a48b-ea70a5dbf3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:10.160250 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf9e54-1b65-4f84-a48b-ea70a5dbf3f3 admin admin] PATH_INFO: `/v3/users/ed98ed05742741b78c11f3483c786588` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:10.162484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bf9e54-1b65-4f84-a48b-ea70a5dbf3f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed98ed05742741b78c11f3483c786588)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:10.166537 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bf9e54-1b65-4f84-a48b-ea70a5dbf3f3 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:10.286653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-25bf9e54-1b65-4f84-a48b-ea70a5dbf3f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed98ed05742741b78c11f3483c786588'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:10.300785 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-25bf9e54-1b65-4f84-a48b-ea70a5dbf3f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed98ed05742741b78c11f3483c786588'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:10.326807 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-25bf9e54-1b65-4f84-a48b-ea70a5dbf3f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed98ed05742741b78c11f3483c786588'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:10.327986 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-25bf9e54-1b65-4f84-a48b-ea70a5dbf3f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed98ed05742741b78c11f3483c786588'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:10.338919 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-25bf9e54-1b65-4f84-a48b-ea70a5dbf3f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed98ed05742741b78c11f3483c786588'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:10.346126 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1112/2206] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:14:09 2022] DELETE /identity/v3/users/ed98ed05742741b78c11f3483c786588 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:14:10.350851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088aafe5-c03b-43a9-939b-ead83283fa0c None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:10.351454 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088aafe5-c03b-43a9-939b-ead83283fa0c None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:10.351454 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088aafe5-c03b-43a9-939b-ead83283fa0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:10.372716 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-088aafe5-c03b-43a9-939b-ead83283fa0c None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:14:10.374030 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-088aafe5-c03b-43a9-939b-ead83283fa0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:10.453091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1095/2207] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:14:10.467563 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf9e54-1b65-4f84-a48b-ea70a5dbf3f3 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:10.469887 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-25bf9e54-1b65-4f84-a48b-ea70a5dbf3f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:10.565850 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf9e54-1b65-4f84-a48b-ea70a5dbf3f3 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:10.569970 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1726ffcd-3a13-41fd-806d-0a6cf249ce82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:10.570613 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726ffcd-3a13-41fd-806d-0a6cf249ce82 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:10.570903 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726ffcd-3a13-41fd-806d-0a6cf249ce82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:10.571173 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726ffcd-3a13-41fd-806d-0a6cf249ce82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:10.583084 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1726ffcd-3a13-41fd-806d-0a6cf249ce82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:10.584713 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1726ffcd-3a13-41fd-806d-0a6cf249ce82 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:10.596412 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1113/2208] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:14:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:14:11.054394 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088aafe5-c03b-43a9-939b-ead83283fa0c None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:11.057208 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-088aafe5-c03b-43a9-939b-ead83283fa0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:11.082001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088aafe5-c03b-43a9-939b-ead83283fa0c None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:11.090426 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0ed73-6c2f-4880-91c3-ea62ecfcca9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:11.091441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0ed73-6c2f-4880-91c3-ea62ecfcca9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:11.091782 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0ed73-6c2f-4880-91c3-ea62ecfcca9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:11.092120 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0ed73-6c2f-4880-91c3-ea62ecfcca9b admin admin] PATH_INFO: `/v3/projects/4f332abb00db4704a832fab5f85615b6` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:11.094001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0ed73-6c2f-4880-91c3-ea62ecfcca9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f332abb00db4704a832fab5f85615b6)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:11.098788 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0ed73-6c2f-4880-91c3-ea62ecfcca9b admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:11.165712 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-f0f0ed73-6c2f-4880-91c3-ea62ecfcca9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f332abb00db4704a832fab5f85615b6', 'request_id': 'req-f0f0ed73-6c2f-4880-91c3-ea62ecfcca9b'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:11.174961 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-f0f0ed73-6c2f-4880-91c3-ea62ecfcca9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f332abb00db4704a832fab5f85615b6', 'request_id': 'req-f0f0ed73-6c2f-4880-91c3-ea62ecfcca9b'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:11.183132 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-f0f0ed73-6c2f-4880-91c3-ea62ecfcca9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f332abb00db4704a832fab5f85615b6', 'request_id': 'req-f0f0ed73-6c2f-4880-91c3-ea62ecfcca9b'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:11.200510 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-f0f0ed73-6c2f-4880-91c3-ea62ecfcca9b admin admin] The token cache is being invalidate because project 4f332abb00db4704a832fab5f85615b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:14:11.200994 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-f0f0ed73-6c2f-4880-91c3-ea62ecfcca9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:11.202197 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-f0f0ed73-6c2f-4880-91c3-ea62ecfcca9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:11.204055 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1096/2209] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:14:11 2022] DELETE /identity/v3/projects/4f332abb00db4704a832fab5f85615b6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:14:11.208957 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1726ffcd-3a13-41fd-806d-0a6cf249ce82 service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:11.219732 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1726ffcd-3a13-41fd-806d-0a6cf249ce82 service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:11.243865 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:14:11.318032 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e063ca-55f5-47af-99b8-80bbbd003946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:11.319117 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e063ca-55f5-47af-99b8-80bbbd003946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:11.319612 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e063ca-55f5-47af-99b8-80bbbd003946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:11.320292 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e063ca-55f5-47af-99b8-80bbbd003946 admin admin] PATH_INFO: `/v3/users/7b69d3c954394722a4d5174ebe079c2d` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:11.323343 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e063ca-55f5-47af-99b8-80bbbd003946 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b69d3c954394722a4d5174ebe079c2d)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:11.327062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e063ca-55f5-47af-99b8-80bbbd003946 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:11.419357 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-57e063ca-55f5-47af-99b8-80bbbd003946 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b69d3c954394722a4d5174ebe079c2d'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:11.437812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-57e063ca-55f5-47af-99b8-80bbbd003946 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b69d3c954394722a4d5174ebe079c2d'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:11.457735 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-57e063ca-55f5-47af-99b8-80bbbd003946 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b69d3c954394722a4d5174ebe079c2d'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:11.458666 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-57e063ca-55f5-47af-99b8-80bbbd003946 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b69d3c954394722a4d5174ebe079c2d'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:11.468360 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-57e063ca-55f5-47af-99b8-80bbbd003946 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b69d3c954394722a4d5174ebe079c2d'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:11.475749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1114/2210] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:14:11 2022] DELETE /identity/v3/users/7b69d3c954394722a4d5174ebe079c2d => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:14:11.945885 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0ed73-6c2f-4880-91c3-ea62ecfcca9b admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:11.948890 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f0f0ed73-6c2f-4880-91c3-ea62ecfcca9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:11.979505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0ed73-6c2f-4880-91c3-ea62ecfcca9b admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:12.084479 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120e121-6203-4a4d-8af3-3fc28d3a6926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:12.085488 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120e121-6203-4a4d-8af3-3fc28d3a6926 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:12.086126 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120e121-6203-4a4d-8af3-3fc28d3a6926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:12.086545 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120e121-6203-4a4d-8af3-3fc28d3a6926 admin admin] PATH_INFO: `/v3/projects/4158922c8df6471795b40544c774dccf` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:12.088734 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b120e121-6203-4a4d-8af3-3fc28d3a6926 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4158922c8df6471795b40544c774dccf)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:12.093777 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b120e121-6203-4a4d-8af3-3fc28d3a6926 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:12.140878 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:14:12.184594 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-b120e121-6203-4a4d-8af3-3fc28d3a6926 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4158922c8df6471795b40544c774dccf', 'request_id': 'req-b120e121-6203-4a4d-8af3-3fc28d3a6926'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:12.201427 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-b120e121-6203-4a4d-8af3-3fc28d3a6926 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4158922c8df6471795b40544c774dccf', 'request_id': 'req-b120e121-6203-4a4d-8af3-3fc28d3a6926'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:12.211382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-b120e121-6203-4a4d-8af3-3fc28d3a6926 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4158922c8df6471795b40544c774dccf', 'request_id': 'req-b120e121-6203-4a4d-8af3-3fc28d3a6926'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:12.236387 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-b120e121-6203-4a4d-8af3-3fc28d3a6926 admin admin] The token cache is being invalidate because project 4158922c8df6471795b40544c774dccf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:14:12.236845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-b120e121-6203-4a4d-8af3-3fc28d3a6926 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:12.237730 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-b120e121-6203-4a4d-8af3-3fc28d3a6926 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:12.239559 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1097/2211] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:14:11 2022] DELETE /identity/v3/projects/4158922c8df6471795b40544c774dccf => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:14:12.247084 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5139e06-2b9b-472f-881e-798beeb72a45 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:12.247871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5139e06-2b9b-472f-881e-798beeb72a45 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:12.248044 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5139e06-2b9b-472f-881e-798beeb72a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:12.283062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-e5139e06-2b9b-472f-881e-798beeb72a45 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:14:12.285169 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-e5139e06-2b9b-472f-881e-798beeb72a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:12.405214 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1115/2212] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:14:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:14:12.418627 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64e046-4a8d-4a34-bf3c-066063c55c8b None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:12.419070 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64e046-4a8d-4a34-bf3c-066063c55c8b None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:12.419494 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64e046-4a8d-4a34-bf3c-066063c55c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:12.449725 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-8e64e046-4a8d-4a34-bf3c-066063c55c8b None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:14:12.450983 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-8e64e046-4a8d-4a34-bf3c-066063c55c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:12.460456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1098/2213] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:14:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:14:12.466125 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5139e06-2b9b-472f-881e-798beeb72a45 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:12.481234 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5139e06-2b9b-472f-881e-798beeb72a45 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:12.487833 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee2dab-ff74-4806-abf1-59927aa92deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:12.493546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee2dab-ff74-4806-abf1-59927aa92deb admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:12.493546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee2dab-ff74-4806-abf1-59927aa92deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:12.493546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee2dab-ff74-4806-abf1-59927aa92deb admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:12.493546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee2dab-ff74-4806-abf1-59927aa92deb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:14:12.493546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee2dab-ff74-4806-abf1-59927aa92deb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:12.493546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee2dab-ff74-4806-abf1-59927aa92deb admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:12.511102 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1116/2214] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:14:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:14:12.516969 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64e046-4a8d-4a34-bf3c-066063c55c8b None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:12.529392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64e046-4a8d-4a34-bf3c-066063c55c8b None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:12.536399 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117fd98-8b7b-434d-b602-109edb3db6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:12.537507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117fd98-8b7b-434d-b602-109edb3db6b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:12.537891 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117fd98-8b7b-434d-b602-109edb3db6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:12.538496 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117fd98-8b7b-434d-b602-109edb3db6b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:12.539486 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c117fd98-8b7b-434d-b602-109edb3db6b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:12.542285 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c117fd98-8b7b-434d-b602-109edb3db6b7 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:12.587383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1099/2215] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:14:12 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:14:12.592939 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee2dab-ff74-4806-abf1-59927aa92deb admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:12.612203 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee2dab-ff74-4806-abf1-59927aa92deb admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:12.656143 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:14:12.696751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee548796-df97-49d5-a362-7824d925cbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:12.697967 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee548796-df97-49d5-a362-7824d925cbfa admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:12.698310 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee548796-df97-49d5-a362-7824d925cbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:12.698630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee548796-df97-49d5-a362-7824d925cbfa admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:12.699332 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee548796-df97-49d5-a362-7824d925cbfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:12.701636 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee548796-df97-49d5-a362-7824d925cbfa admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:12.734243 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1117/2216] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:14:12 2022] POST /identity/v3/users => generated 287 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:14:12.739759 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117fd98-8b7b-434d-b602-109edb3db6b7 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:12.758535 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117fd98-8b7b-434d-b602-109edb3db6b7 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:12.766745 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51a811-c81f-46c0-863e-02b8c61a5f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:12.767739 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51a811-c81f-46c0-863e-02b8c61a5f67 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:12.768198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51a811-c81f-46c0-863e-02b8c61a5f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:12.768663 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51a811-c81f-46c0-863e-02b8c61a5f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:12.769844 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51a811-c81f-46c0-863e-02b8c61a5f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:12.772885 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51a811-c81f-46c0-863e-02b8c61a5f67 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:12.790179 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1100/2217] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:14:12.797861 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee548796-df97-49d5-a362-7824d925cbfa admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:12.810271 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee548796-df97-49d5-a362-7824d925cbfa admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:12.817416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca1635-e42e-406c-826b-e055f14a6d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:12.818401 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca1635-e42e-406c-826b-e055f14a6d37 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:12.818851 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca1635-e42e-406c-826b-e055f14a6d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:12.819900 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca1635-e42e-406c-826b-e055f14a6d37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:12.821307 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca1635-e42e-406c-826b-e055f14a6d37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:12.823477 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca1635-e42e-406c-826b-e055f14a6d37 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:12.837235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1118/2218] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:14:12.844019 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51a811-c81f-46c0-863e-02b8c61a5f67 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:12.861886 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51a811-c81f-46c0-863e-02b8c61a5f67 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:12.870197 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6637abb-4fac-45f3-9f9d-e7a6d144f879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:12.871336 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6637abb-4fac-45f3-9f9d-e7a6d144f879 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:12.871810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6637abb-4fac-45f3-9f9d-e7a6d144f879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:12.872299 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6637abb-4fac-45f3-9f9d-e7a6d144f879 admin admin] PATH_INFO: `/v3/projects/076251c420374fd4b9d325b4545b5db8/users/7816509ce2984ad2a134d87d8dacfa87/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:12.899902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6637abb-4fac-45f3-9f9d-e7a6d144f879 admin admin] RBAC: Authorizing `identity:create_grant(project_id=076251c420374fd4b9d325b4545b5db8, user_id=7816509ce2984ad2a134d87d8dacfa87, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:12.904988 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6637abb-4fac-45f3-9f9d-e7a6d144f879 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:12.924094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1101/2219] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:14:12 2022] PUT /identity/v3/projects/076251c420374fd4b9d325b4545b5db8/users/7816509ce2984ad2a134d87d8dacfa87/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:14:13.035598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a373135-5913-4e31-b090-837389f2332d None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:13.036056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a373135-5913-4e31-b090-837389f2332d None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:13.036508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a373135-5913-4e31-b090-837389f2332d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:13.074331 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-8a373135-5913-4e31-b090-837389f2332d None None] MFA Rules not processed for user `7816509ce2984ad2a134d87d8dacfa87`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:14:13.075314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-8a373135-5913-4e31-b090-837389f2332d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:13.195579 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1119/2220] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:14:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:14:13.209191 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6637abb-4fac-45f3-9f9d-e7a6d144f879 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:13.211952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b6637abb-4fac-45f3-9f9d-e7a6d144f879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:13.325093 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6637abb-4fac-45f3-9f9d-e7a6d144f879 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:13.332526 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae11e4-7854-4ec2-a2a4-88b83622eb0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:13.333712 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae11e4-7854-4ec2-a2a4-88b83622eb0a service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:13.334189 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae11e4-7854-4ec2-a2a4-88b83622eb0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:13.334642 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae11e4-7854-4ec2-a2a4-88b83622eb0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:13.351196 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae11e4-7854-4ec2-a2a4-88b83622eb0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:13.353805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae11e4-7854-4ec2-a2a4-88b83622eb0a service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:13.371341 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1102/2221] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:14:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:14:13.680241 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a373135-5913-4e31-b090-837389f2332d None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:13.682461 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-8a373135-5913-4e31-b090-837389f2332d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:13.774022 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a373135-5913-4e31-b090-837389f2332d None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:13.779467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c83cb-f4d7-4c64-ab58-ec8b7f99a8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:13.780146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c83cb-f4d7-4c64-ab58-ec8b7f99a8f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:13.780431 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c83cb-f4d7-4c64-ab58-ec8b7f99a8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:13.780693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c83cb-f4d7-4c64-ab58-ec8b7f99a8f9 admin admin] PATH_INFO: `/v3/users/e8aafc81a7754e839b45deef6c16cffe` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:13.781966 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c83cb-f4d7-4c64-ab58-ec8b7f99a8f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8aafc81a7754e839b45deef6c16cffe)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:13.783918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c83cb-f4d7-4c64-ab58-ec8b7f99a8f9 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:13.861385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:14:13.870633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e66c83cb-f4d7-4c64-ab58-ec8b7f99a8f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8aafc81a7754e839b45deef6c16cffe'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:13.889689 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e66c83cb-f4d7-4c64-ab58-ec8b7f99a8f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8aafc81a7754e839b45deef6c16cffe'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:13.917649 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e66c83cb-f4d7-4c64-ab58-ec8b7f99a8f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8aafc81a7754e839b45deef6c16cffe'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:13.918534 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e66c83cb-f4d7-4c64-ab58-ec8b7f99a8f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8aafc81a7754e839b45deef6c16cffe'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:13.930303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e66c83cb-f4d7-4c64-ab58-ec8b7f99a8f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8aafc81a7754e839b45deef6c16cffe'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:13.940100 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1120/2222] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:14:13 2022] DELETE /identity/v3/users/e8aafc81a7754e839b45deef6c16cffe => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:14:13.946030 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af463c2a-2f4c-4161-a516-ed9b04b19622 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:13.946525 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af463c2a-2f4c-4161-a516-ed9b04b19622 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:13.946986 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af463c2a-2f4c-4161-a516-ed9b04b19622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:13.975022 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-af463c2a-2f4c-4161-a516-ed9b04b19622 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:14:13.976283 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-af463c2a-2f4c-4161-a516-ed9b04b19622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:14.030183 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:14:14.054256 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1103/2223] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:14:14.064758 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c83cb-f4d7-4c64-ab58-ec8b7f99a8f9 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:14.066271 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-e66c83cb-f4d7-4c64-ab58-ec8b7f99a8f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:14.130680 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c83cb-f4d7-4c64-ab58-ec8b7f99a8f9 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:14.134471 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743be77f-f610-4234-aad2-43ec669a15ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:14.135103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743be77f-f610-4234-aad2-43ec669a15ad service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:14.135407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743be77f-f610-4234-aad2-43ec669a15ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:14.135676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743be77f-f610-4234-aad2-43ec669a15ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:14.146010 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743be77f-f610-4234-aad2-43ec669a15ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:14.147482 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743be77f-f610-4234-aad2-43ec669a15ad service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:14.158564 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1121/2224] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:14:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:14:14.512745 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af463c2a-2f4c-4161-a516-ed9b04b19622 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:14.514887 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-af463c2a-2f4c-4161-a516-ed9b04b19622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:14.530779 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af463c2a-2f4c-4161-a516-ed9b04b19622 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:14.536467 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52414b83-f80f-4026-816c-163de990e7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:14.537540 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52414b83-f80f-4026-816c-163de990e7cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:14.537990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52414b83-f80f-4026-816c-163de990e7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:14.538472 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52414b83-f80f-4026-816c-163de990e7cb admin admin] PATH_INFO: `/v3/projects/554e4c2a862b447f859321337f3135e6` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:14.539978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52414b83-f80f-4026-816c-163de990e7cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=554e4c2a862b447f859321337f3135e6)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:14.542213 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52414b83-f80f-4026-816c-163de990e7cb admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:14.632950 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-52414b83-f80f-4026-816c-163de990e7cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '554e4c2a862b447f859321337f3135e6', 'request_id': 'req-52414b83-f80f-4026-816c-163de990e7cb'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:14.649504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-52414b83-f80f-4026-816c-163de990e7cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '554e4c2a862b447f859321337f3135e6', 'request_id': 'req-52414b83-f80f-4026-816c-163de990e7cb'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:14.664487 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-52414b83-f80f-4026-816c-163de990e7cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '554e4c2a862b447f859321337f3135e6', 'request_id': 'req-52414b83-f80f-4026-816c-163de990e7cb'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:14.683661 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-52414b83-f80f-4026-816c-163de990e7cb admin admin] The token cache is being invalidate because project 554e4c2a862b447f859321337f3135e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:14:14.683948 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-52414b83-f80f-4026-816c-163de990e7cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:14.684809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-52414b83-f80f-4026-816c-163de990e7cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:14.686545 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1104/2225] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:14:14 2022] DELETE /identity/v3/projects/554e4c2a862b447f859321337f3135e6 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:14:14.692426 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382af9f-50fb-487d-875f-e2a41acccd26 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:14.693295 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382af9f-50fb-487d-875f-e2a41acccd26 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:14.693468 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382af9f-50fb-487d-875f-e2a41acccd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:14.723593 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-b382af9f-50fb-487d-875f-e2a41acccd26 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:14:14.725426 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-b382af9f-50fb-487d-875f-e2a41acccd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:14.828967 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1122/2226] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:14:14.839652 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7a491-be81-496d-b444-20b6e3bceb9a None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:14.840227 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7a491-be81-496d-b444-20b6e3bceb9a None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:14.840704 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7a491-be81-496d-b444-20b6e3bceb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:14.875120 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-ece7a491-be81-496d-b444-20b6e3bceb9a None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:14:14.877072 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-ece7a491-be81-496d-b444-20b6e3bceb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:14.889781 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1105/2227] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:14:14.894400 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382af9f-50fb-487d-875f-e2a41acccd26 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:14.905046 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382af9f-50fb-487d-875f-e2a41acccd26 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:14.910020 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434a2b9-cd38-4d1c-a629-f9c01b007697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:14.910664 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434a2b9-cd38-4d1c-a629-f9c01b007697 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:14.911007 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434a2b9-cd38-4d1c-a629-f9c01b007697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:14.911311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434a2b9-cd38-4d1c-a629-f9c01b007697 admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:14.911974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0434a2b9-cd38-4d1c-a629-f9c01b007697 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:14:14.912278 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0434a2b9-cd38-4d1c-a629-f9c01b007697 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:14.913962 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0434a2b9-cd38-4d1c-a629-f9c01b007697 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:14.931526 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1123/2228] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:14:14.937393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece7a491-be81-496d-b444-20b6e3bceb9a None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:14.950237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece7a491-be81-496d-b444-20b6e3bceb9a None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:14.956553 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1c646-70ea-427e-b9e7-e0e00c98fa09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:14.957414 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1c646-70ea-427e-b9e7-e0e00c98fa09 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:14.957783 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1c646-70ea-427e-b9e7-e0e00c98fa09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:14.958238 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1c646-70ea-427e-b9e7-e0e00c98fa09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:14.959320 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b1c646-70ea-427e-b9e7-e0e00c98fa09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:14.963147 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b1c646-70ea-427e-b9e7-e0e00c98fa09 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:15.007388 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1106/2229] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:14:14 2022] POST /identity/v3/projects => generated 363 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:14:15.012607 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434a2b9-cd38-4d1c-a629-f9c01b007697 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:15.029492 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434a2b9-cd38-4d1c-a629-f9c01b007697 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:15.128409 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:14:15.153902 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b6db1-ac21-4527-9400-4c6b4b9ca5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:15.154688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b6db1-ac21-4527-9400-4c6b4b9ca5b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:15.154968 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b6db1-ac21-4527-9400-4c6b4b9ca5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:15.155289 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b6db1-ac21-4527-9400-4c6b4b9ca5b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:15.155926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b6db1-ac21-4527-9400-4c6b4b9ca5b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:15.158165 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b6db1-ac21-4527-9400-4c6b4b9ca5b8 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:15.186921 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1124/2230] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:14:15 2022] POST /identity/v3/users => generated 282 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:14:15.191346 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1c646-70ea-427e-b9e7-e0e00c98fa09 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:15.204112 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1c646-70ea-427e-b9e7-e0e00c98fa09 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:15.209906 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d6fbe-27a5-4eef-89ff-d999a0821369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:15.210740 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d6fbe-27a5-4eef-89ff-d999a0821369 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:15.211198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d6fbe-27a5-4eef-89ff-d999a0821369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:15.211557 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d6fbe-27a5-4eef-89ff-d999a0821369 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:15.212417 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d6fbe-27a5-4eef-89ff-d999a0821369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:15.214297 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d6fbe-27a5-4eef-89ff-d999a0821369 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:15.226825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1107/2231] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:14:15.231102 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b6db1-ac21-4527-9400-4c6b4b9ca5b8 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:15.243737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b6db1-ac21-4527-9400-4c6b4b9ca5b8 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:15.251325 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9879ab6-587f-4187-979b-b6d7b492fb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:15.253090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9879ab6-587f-4187-979b-b6d7b492fb39 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:15.253633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9879ab6-587f-4187-979b-b6d7b492fb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:15.254034 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9879ab6-587f-4187-979b-b6d7b492fb39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:15.255208 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9879ab6-587f-4187-979b-b6d7b492fb39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:15.258304 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9879ab6-587f-4187-979b-b6d7b492fb39 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:15.273728 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1125/2232] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:14:15.278509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d6fbe-27a5-4eef-89ff-d999a0821369 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:15.292294 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d6fbe-27a5-4eef-89ff-d999a0821369 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:15.298799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a7a7c-0f99-4839-b35c-6b798111af27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:15.299741 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a7a7c-0f99-4839-b35c-6b798111af27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:15.299998 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a7a7c-0f99-4839-b35c-6b798111af27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:15.300369 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a7a7c-0f99-4839-b35c-6b798111af27 admin admin] PATH_INFO: `/v3/projects/dc1eb61863704af88224b447d96abd52/users/6c58cf4b3ecc4881ba1fe8ae014c8aa9/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:15.318275 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a7a7c-0f99-4839-b35c-6b798111af27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc1eb61863704af88224b447d96abd52, user_id=6c58cf4b3ecc4881ba1fe8ae014c8aa9, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:15.321247 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a7a7c-0f99-4839-b35c-6b798111af27 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:15.338646 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1108/2233] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:14:15 2022] PUT /identity/v3/projects/dc1eb61863704af88224b447d96abd52/users/6c58cf4b3ecc4881ba1fe8ae014c8aa9/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:14:15.351400 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4886c35-2bfa-47ba-90ae-fceaab54128b None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:15.351883 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4886c35-2bfa-47ba-90ae-fceaab54128b None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:15.352312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4886c35-2bfa-47ba-90ae-fceaab54128b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:15.400648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-c4886c35-2bfa-47ba-90ae-fceaab54128b None None] MFA Rules not processed for user `6c58cf4b3ecc4881ba1fe8ae014c8aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:14:15.402971 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-c4886c35-2bfa-47ba-90ae-fceaab54128b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:15.532471 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1126/2234] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:14:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:14:15.549905 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a7a7c-0f99-4839-b35c-6b798111af27 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:15.553806 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-075a7a7c-0f99-4839-b35c-6b798111af27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:15.660437 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a7a7c-0f99-4839-b35c-6b798111af27 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:15.666806 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217c84e-5905-4d76-856e-2313c47917ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:15.667853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217c84e-5905-4d76-856e-2313c47917ae service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:15.668324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217c84e-5905-4d76-856e-2313c47917ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:15.668784 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217c84e-5905-4d76-856e-2313c47917ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:15.685385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d217c84e-5905-4d76-856e-2313c47917ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:15.687134 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d217c84e-5905-4d76-856e-2313c47917ae service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:15.698810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1109/2235] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:14:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:14:18.622925 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4886c35-2bfa-47ba-90ae-fceaab54128b None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:18.632135 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-c4886c35-2bfa-47ba-90ae-fceaab54128b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:18.820615 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4886c35-2bfa-47ba-90ae-fceaab54128b None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:18.832671 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957178e-f2ad-4013-baed-3df323690368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:18.834004 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957178e-f2ad-4013-baed-3df323690368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:18.834808 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957178e-f2ad-4013-baed-3df323690368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:18.835338 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957178e-f2ad-4013-baed-3df323690368 admin admin] PATH_INFO: `/v3/users/8d1150f59b7448d7af330cfbb70d32f8` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:18.838002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957178e-f2ad-4013-baed-3df323690368 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d1150f59b7448d7af330cfbb70d32f8)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:18.846897 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957178e-f2ad-4013-baed-3df323690368 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:18.965491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-0957178e-f2ad-4013-baed-3df323690368 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d1150f59b7448d7af330cfbb70d32f8'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:18.976567 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-0957178e-f2ad-4013-baed-3df323690368 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d1150f59b7448d7af330cfbb70d32f8'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:18.979598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:14:19.000218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-0957178e-f2ad-4013-baed-3df323690368 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d1150f59b7448d7af330cfbb70d32f8'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:19.001106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-0957178e-f2ad-4013-baed-3df323690368 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d1150f59b7448d7af330cfbb70d32f8'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:19.013101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-0957178e-f2ad-4013-baed-3df323690368 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d1150f59b7448d7af330cfbb70d32f8'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:19.037344 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1127/2236] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:14:18 2022] DELETE /identity/v3/users/8d1150f59b7448d7af330cfbb70d32f8 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:14:19.039216 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731262d-4c84-442b-a6cb-3357969a067e None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:19.039971 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731262d-4c84-442b-a6cb-3357969a067e None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:19.040403 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731262d-4c84-442b-a6cb-3357969a067e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:19.078586 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-c731262d-4c84-442b-a6cb-3357969a067e None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:14:19.080627 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c731262d-4c84-442b-a6cb-3357969a067e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:19.090015 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:14:19.188755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1110/2237] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:14:19.202811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957178e-f2ad-4013-baed-3df323690368 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:19.204540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-0957178e-f2ad-4013-baed-3df323690368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:19.305520 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957178e-f2ad-4013-baed-3df323690368 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:19.315234 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3488a87-0ddd-46ad-bb00-5f18afaac5fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:19.316352 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3488a87-0ddd-46ad-bb00-5f18afaac5fd service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:19.316920 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3488a87-0ddd-46ad-bb00-5f18afaac5fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:19.317441 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3488a87-0ddd-46ad-bb00-5f18afaac5fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:19.339554 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3488a87-0ddd-46ad-bb00-5f18afaac5fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:19.342769 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3488a87-0ddd-46ad-bb00-5f18afaac5fd service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:19.359719 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1128/2238] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:14:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:14:19.760013 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c731262d-4c84-442b-a6cb-3357969a067e None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:19.762377 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c731262d-4c84-442b-a6cb-3357969a067e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:19.785962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c731262d-4c84-442b-a6cb-3357969a067e None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:19.794917 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792f437-80eb-4ae4-96d5-422dccba9a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:19.796088 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792f437-80eb-4ae4-96d5-422dccba9a03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:19.796789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792f437-80eb-4ae4-96d5-422dccba9a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:19.797417 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792f437-80eb-4ae4-96d5-422dccba9a03 admin admin] PATH_INFO: `/v3/projects/5afb78ce354d4279b9926e1f7718846a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:19.799498 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d792f437-80eb-4ae4-96d5-422dccba9a03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5afb78ce354d4279b9926e1f7718846a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:19.803488 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d792f437-80eb-4ae4-96d5-422dccba9a03 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:19.889549 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d792f437-80eb-4ae4-96d5-422dccba9a03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5afb78ce354d4279b9926e1f7718846a', 'request_id': 'req-d792f437-80eb-4ae4-96d5-422dccba9a03'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:19.902862 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d792f437-80eb-4ae4-96d5-422dccba9a03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5afb78ce354d4279b9926e1f7718846a', 'request_id': 'req-d792f437-80eb-4ae4-96d5-422dccba9a03'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:19.919511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d792f437-80eb-4ae4-96d5-422dccba9a03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5afb78ce354d4279b9926e1f7718846a', 'request_id': 'req-d792f437-80eb-4ae4-96d5-422dccba9a03'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:19.949324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d792f437-80eb-4ae4-96d5-422dccba9a03 admin admin] The token cache is being invalidate because project 5afb78ce354d4279b9926e1f7718846a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:14:19.950623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d792f437-80eb-4ae4-96d5-422dccba9a03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:19.952133 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d792f437-80eb-4ae4-96d5-422dccba9a03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:19.955775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1111/2239] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:14:19 2022] DELETE /identity/v3/projects/5afb78ce354d4279b9926e1f7718846a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:14:19.962059 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3488a87-0ddd-46ad-bb00-5f18afaac5fd service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:19.983679 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3488a87-0ddd-46ad-bb00-5f18afaac5fd service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:20.116941 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c3d74-8ad4-4a3e-9d59-d52502ef1d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:20.116941 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c3d74-8ad4-4a3e-9d59-d52502ef1d41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:20.116941 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c3d74-8ad4-4a3e-9d59-d52502ef1d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:20.116941 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c3d74-8ad4-4a3e-9d59-d52502ef1d41 admin admin] PATH_INFO: `/v3/users/cdc9506a31344f98a914b91612d1cb72` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:20.117497 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c3d74-8ad4-4a3e-9d59-d52502ef1d41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdc9506a31344f98a914b91612d1cb72)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:20.129176 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c3d74-8ad4-4a3e-9d59-d52502ef1d41 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:20.214543 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-bd1c3d74-8ad4-4a3e-9d59-d52502ef1d41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdc9506a31344f98a914b91612d1cb72'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:20.227590 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-bd1c3d74-8ad4-4a3e-9d59-d52502ef1d41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdc9506a31344f98a914b91612d1cb72'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:20.265703 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-bd1c3d74-8ad4-4a3e-9d59-d52502ef1d41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdc9506a31344f98a914b91612d1cb72'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:20.267156 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-bd1c3d74-8ad4-4a3e-9d59-d52502ef1d41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdc9506a31344f98a914b91612d1cb72'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:20.278493 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:14:20.282080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-bd1c3d74-8ad4-4a3e-9d59-d52502ef1d41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdc9506a31344f98a914b91612d1cb72'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:20.292600 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1129/2240] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:14:19 2022] DELETE /identity/v3/users/cdc9506a31344f98a914b91612d1cb72 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:14:21.093660 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792f437-80eb-4ae4-96d5-422dccba9a03 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:21.097310 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-d792f437-80eb-4ae4-96d5-422dccba9a03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:21.122513 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792f437-80eb-4ae4-96d5-422dccba9a03 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:21.232156 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2aa1e2-49aa-4028-a570-db6da383533c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:21.232807 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2aa1e2-49aa-4028-a570-db6da383533c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:21.233080 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2aa1e2-49aa-4028-a570-db6da383533c admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:21.233422 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2aa1e2-49aa-4028-a570-db6da383533c admin admin] PATH_INFO: `/v3/projects/4e72d2e0e625441986ed00874e3af0ea` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:21.234639 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2aa1e2-49aa-4028-a570-db6da383533c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e72d2e0e625441986ed00874e3af0ea)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:21.236592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2aa1e2-49aa-4028-a570-db6da383533c admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:21.304539 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:14:21.331141 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-7e2aa1e2-49aa-4028-a570-db6da383533c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e72d2e0e625441986ed00874e3af0ea', 'request_id': 'req-7e2aa1e2-49aa-4028-a570-db6da383533c'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:21.345578 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-7e2aa1e2-49aa-4028-a570-db6da383533c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e72d2e0e625441986ed00874e3af0ea', 'request_id': 'req-7e2aa1e2-49aa-4028-a570-db6da383533c'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:21.362172 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-7e2aa1e2-49aa-4028-a570-db6da383533c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e72d2e0e625441986ed00874e3af0ea', 'request_id': 'req-7e2aa1e2-49aa-4028-a570-db6da383533c'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:21.377970 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-7e2aa1e2-49aa-4028-a570-db6da383533c admin admin] The token cache is being invalidate because project 4e72d2e0e625441986ed00874e3af0ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:14:21.378308 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-7e2aa1e2-49aa-4028-a570-db6da383533c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:21.379771 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-7e2aa1e2-49aa-4028-a570-db6da383533c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:21.381235 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1112/2241] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:14:21 2022] DELETE /identity/v3/projects/4e72d2e0e625441986ed00874e3af0ea => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:14:21.386071 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72109e8f-6136-48ac-9a6f-bf2682f2e701 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:21.387020 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72109e8f-6136-48ac-9a6f-bf2682f2e701 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:21.387523 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72109e8f-6136-48ac-9a6f-bf2682f2e701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:21.421569 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-72109e8f-6136-48ac-9a6f-bf2682f2e701 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:14:21.422925 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-72109e8f-6136-48ac-9a6f-bf2682f2e701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:21.534408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1130/2242] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:14:21.545995 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e0ab9-c12d-49ca-97c8-ee006cd71262 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:21.546424 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e0ab9-c12d-49ca-97c8-ee006cd71262 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:21.546794 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e0ab9-c12d-49ca-97c8-ee006cd71262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:21.570768 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-104e0ab9-c12d-49ca-97c8-ee006cd71262 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:14:21.572031 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-104e0ab9-c12d-49ca-97c8-ee006cd71262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:21.581886 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1113/2243] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:14:21.586235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72109e8f-6136-48ac-9a6f-bf2682f2e701 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:21.597968 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72109e8f-6136-48ac-9a6f-bf2682f2e701 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:21.603046 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f43b97-dba0-4464-9ee6-998a0385beea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:21.603679 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f43b97-dba0-4464-9ee6-998a0385beea admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:21.603989 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f43b97-dba0-4464-9ee6-998a0385beea admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:21.604252 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f43b97-dba0-4464-9ee6-998a0385beea admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:21.604923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f43b97-dba0-4464-9ee6-998a0385beea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:14:21.605261 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f43b97-dba0-4464-9ee6-998a0385beea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:21.607148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f43b97-dba0-4464-9ee6-998a0385beea admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:21.621291 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1131/2244] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:14:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:14:21.626120 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e0ab9-c12d-49ca-97c8-ee006cd71262 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:21.637500 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e0ab9-c12d-49ca-97c8-ee006cd71262 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:21.643653 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d836c8-2452-4530-bfcb-baab10940e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:21.644393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d836c8-2452-4530-bfcb-baab10940e30 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:21.644725 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d836c8-2452-4530-bfcb-baab10940e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:21.645267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d836c8-2452-4530-bfcb-baab10940e30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:21.646039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d836c8-2452-4530-bfcb-baab10940e30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:21.648265 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d836c8-2452-4530-bfcb-baab10940e30 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:21.682703 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1114/2245] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:14:21 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:14:21.687278 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f43b97-dba0-4464-9ee6-998a0385beea admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:21.700727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f43b97-dba0-4464-9ee6-998a0385beea admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:21.781233 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69748a5c-2c5d-4c02-a55a-5d71e1b19313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:21.782196 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69748a5c-2c5d-4c02-a55a-5d71e1b19313 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:21.782974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69748a5c-2c5d-4c02-a55a-5d71e1b19313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:21.783091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69748a5c-2c5d-4c02-a55a-5d71e1b19313 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:21.784179 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69748a5c-2c5d-4c02-a55a-5d71e1b19313 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:21.787823 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69748a5c-2c5d-4c02-a55a-5d71e1b19313 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:21.827883 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1132/2246] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:14:21 2022] POST /identity/v3/users => generated 289 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:14:21.829517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d836c8-2452-4530-bfcb-baab10940e30 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:21.841884 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d836c8-2452-4530-bfcb-baab10940e30 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:21.848205 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d3a52-fc4c-4012-b71e-4fa6550a000f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:21.848922 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d3a52-fc4c-4012-b71e-4fa6550a000f admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:21.849416 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d3a52-fc4c-4012-b71e-4fa6550a000f admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:21.849735 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d3a52-fc4c-4012-b71e-4fa6550a000f admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:21.850759 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d3a52-fc4c-4012-b71e-4fa6550a000f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:21.852603 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d3a52-fc4c-4012-b71e-4fa6550a000f admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:21.868227 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1115/2247] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:14:21.869621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69748a5c-2c5d-4c02-a55a-5d71e1b19313 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:21.881043 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69748a5c-2c5d-4c02-a55a-5d71e1b19313 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:21.886767 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18128b-dbcd-47ed-ab02-be420b0f2fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:21.887402 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18128b-dbcd-47ed-ab02-be420b0f2fdd admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:21.887678 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18128b-dbcd-47ed-ab02-be420b0f2fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:21.888029 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18128b-dbcd-47ed-ab02-be420b0f2fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:21.888871 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f18128b-dbcd-47ed-ab02-be420b0f2fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:21.890733 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f18128b-dbcd-47ed-ab02-be420b0f2fdd admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:21.908112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1133/2248] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:14:21.913065 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d3a52-fc4c-4012-b71e-4fa6550a000f admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:21.931333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d3a52-fc4c-4012-b71e-4fa6550a000f admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:21.938459 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3d4c8-03ab-4723-a211-a5d9e559006a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:21.939178 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3d4c8-03ab-4723-a211-a5d9e559006a admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:21.939489 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3d4c8-03ab-4723-a211-a5d9e559006a admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:21.939722 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3d4c8-03ab-4723-a211-a5d9e559006a admin admin] PATH_INFO: `/v3/projects/bd0ba18603f74d278d289ccf43959a18/users/c6fdb6b74f574a54aff5b59032a61cf1/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:21.967441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec3d4c8-03ab-4723-a211-a5d9e559006a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd0ba18603f74d278d289ccf43959a18, user_id=c6fdb6b74f574a54aff5b59032a61cf1, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:21.970784 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec3d4c8-03ab-4723-a211-a5d9e559006a admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:21.990442 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1116/2249] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:14:21 2022] PUT /identity/v3/projects/bd0ba18603f74d278d289ccf43959a18/users/c6fdb6b74f574a54aff5b59032a61cf1/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:14:22.010995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073aaf88-3b66-4d4f-b8f6-a47c3eb9d1a8 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:22.011287 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073aaf88-3b66-4d4f-b8f6-a47c3eb9d1a8 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:22.011544 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073aaf88-3b66-4d4f-b8f6-a47c3eb9d1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:22.050095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-073aaf88-3b66-4d4f-b8f6-a47c3eb9d1a8 None None] MFA Rules not processed for user `c6fdb6b74f574a54aff5b59032a61cf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:14:22.052038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-073aaf88-3b66-4d4f-b8f6-a47c3eb9d1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:22.143123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:14:22.165065 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1134/2250] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:14:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:14:22.170262 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3d4c8-03ab-4723-a211-a5d9e559006a admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:22.245772 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3d4c8-03ab-4723-a211-a5d9e559006a admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:22.256156 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3cc008-35c4-4b68-a5e9-74c8f54ff486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:22.256799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cc008-35c4-4b68-a5e9-74c8f54ff486 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:22.257110 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cc008-35c4-4b68-a5e9-74c8f54ff486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:22.257500 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cc008-35c4-4b68-a5e9-74c8f54ff486 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:22.258183 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3cc008-35c4-4b68-a5e9-74c8f54ff486 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:22.260196 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3cc008-35c4-4b68-a5e9-74c8f54ff486 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:22.302883 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1117/2251] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:14:22 2022] POST /identity/v3/projects => generated 377 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:14:22.307822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073aaf88-3b66-4d4f-b8f6-a47c3eb9d1a8 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:22.320069 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073aaf88-3b66-4d4f-b8f6-a47c3eb9d1a8 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:22.404128 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e87bc-927e-4ec5-9e1d-9d93bf2353e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:22.404829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e87bc-927e-4ec5-9e1d-9d93bf2353e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:22.405126 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e87bc-927e-4ec5-9e1d-9d93bf2353e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:22.405514 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e87bc-927e-4ec5-9e1d-9d93bf2353e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:22.406457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1e87bc-927e-4ec5-9e1d-9d93bf2353e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:22.409126 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1e87bc-927e-4ec5-9e1d-9d93bf2353e4 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:22.444376 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1135/2252] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:14:22 2022] POST /identity/v3/users => generated 295 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:14:22.446053 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3cc008-35c4-4b68-a5e9-74c8f54ff486 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:22.459033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3cc008-35c4-4b68-a5e9-74c8f54ff486 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:22.464964 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a33c047-77aa-49c7-976b-00af570ddda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:22.465644 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33c047-77aa-49c7-976b-00af570ddda5 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:22.465920 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33c047-77aa-49c7-976b-00af570ddda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:22.466185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33c047-77aa-49c7-976b-00af570ddda5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:22.466859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a33c047-77aa-49c7-976b-00af570ddda5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:22.468547 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a33c047-77aa-49c7-976b-00af570ddda5 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:22.482591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1118/2253] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:14:22.487793 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e87bc-927e-4ec5-9e1d-9d93bf2353e4 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:22.500142 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e87bc-927e-4ec5-9e1d-9d93bf2353e4 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:22.506734 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2658cd-06d5-4743-a62c-e9a4c7140e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:22.507437 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2658cd-06d5-4743-a62c-e9a4c7140e24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:22.507838 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2658cd-06d5-4743-a62c-e9a4c7140e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:22.508110 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2658cd-06d5-4743-a62c-e9a4c7140e24 admin admin] PATH_INFO: `/v3/projects/a20323e948dc40d5a8944cf72346fe54/users/522e80f4136849a9a7b1716e8cb36348/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:22.530503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2658cd-06d5-4743-a62c-e9a4c7140e24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a20323e948dc40d5a8944cf72346fe54, user_id=522e80f4136849a9a7b1716e8cb36348, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:22.533348 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2658cd-06d5-4743-a62c-e9a4c7140e24 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:22.556870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1136/2254] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:14:22 2022] PUT /identity/v3/projects/a20323e948dc40d5a8944cf72346fe54/users/522e80f4136849a9a7b1716e8cb36348/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:14:22.561631 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a33c047-77aa-49c7-976b-00af570ddda5 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:22.581193 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a33c047-77aa-49c7-976b-00af570ddda5 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:22.636204 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:14:22.678629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d87149-4bd3-4d1a-8d1b-e93173e226bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:22.679918 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d87149-4bd3-4d1a-8d1b-e93173e226bd admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:22.680294 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d87149-4bd3-4d1a-8d1b-e93173e226bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:22.680662 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d87149-4bd3-4d1a-8d1b-e93173e226bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:22.682151 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d87149-4bd3-4d1a-8d1b-e93173e226bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:22.684836 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d87149-4bd3-4d1a-8d1b-e93173e226bd admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:22.706570 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2658cd-06d5-4743-a62c-e9a4c7140e24 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:22.707731 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1119/2255] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:14:22.721816 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2658cd-06d5-4743-a62c-e9a4c7140e24 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:22.728328 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d203cd-ff4a-4b07-b04f-5f6179aa7fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:22.729098 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d203cd-ff4a-4b07-b04f-5f6179aa7fa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:22.729618 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d203cd-ff4a-4b07-b04f-5f6179aa7fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:22.729968 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d203cd-ff4a-4b07-b04f-5f6179aa7fa4 admin admin] PATH_INFO: `/v3/projects/a20323e948dc40d5a8944cf72346fe54/users/522e80f4136849a9a7b1716e8cb36348/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:22.732447 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d203cd-ff4a-4b07-b04f-5f6179aa7fa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a20323e948dc40d5a8944cf72346fe54, user_id=522e80f4136849a9a7b1716e8cb36348, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:22.735777 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d203cd-ff4a-4b07-b04f-5f6179aa7fa4 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:22.760390 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1137/2256] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:14:22 2022] PUT /identity/v3/projects/a20323e948dc40d5a8944cf72346fe54/users/522e80f4136849a9a7b1716e8cb36348/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:14:22.777795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6f917-9c05-416b-b346-3c0a405f232d None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:22.778416 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6f917-9c05-416b-b346-3c0a405f232d None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:22.778889 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6f917-9c05-416b-b346-3c0a405f232d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:22.831102 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-99a6f917-9c05-416b-b346-3c0a405f232d None None] MFA Rules not processed for user `522e80f4136849a9a7b1716e8cb36348`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:14:22.832782 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-99a6f917-9c05-416b-b346-3c0a405f232d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:22.956210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1120/2257] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:14:22 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:14:22.969806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d203cd-ff4a-4b07-b04f-5f6179aa7fa4 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:22.971769 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-e2d203cd-ff4a-4b07-b04f-5f6179aa7fa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:23.065635 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d203cd-ff4a-4b07-b04f-5f6179aa7fa4 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:23.072671 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e2d014-3ddc-42f2-8f6c-32cfc5d1f07b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:23.073760 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2d014-3ddc-42f2-8f6c-32cfc5d1f07b service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:23.074250 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2d014-3ddc-42f2-8f6c-32cfc5d1f07b service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:23.074713 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2d014-3ddc-42f2-8f6c-32cfc5d1f07b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:23.185371 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e2d014-3ddc-42f2-8f6c-32cfc5d1f07b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:23.188242 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e2d014-3ddc-42f2-8f6c-32cfc5d1f07b service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:23.195575 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:14:23.217561 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1138/2258] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:14:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:14:28.496489 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6f917-9c05-416b-b346-3c0a405f232d None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:28.513098 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6f917-9c05-416b-b346-3c0a405f232d None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:28.520026 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7889a-dbf9-461e-ac21-71ff7b17c27f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:28.521088 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7889a-dbf9-461e-ac21-71ff7b17c27f service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:28.521587 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7889a-dbf9-461e-ac21-71ff7b17c27f service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:28.522054 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7889a-dbf9-461e-ac21-71ff7b17c27f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:28.539828 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c7889a-dbf9-461e-ac21-71ff7b17c27f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:28.542758 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c7889a-dbf9-461e-ac21-71ff7b17c27f service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:28.561923 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1121/2259] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:14:28 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:14:29.223011 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e2d014-3ddc-42f2-8f6c-32cfc5d1f07b service cinder] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:29.225446 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-34e2d014-3ddc-42f2-8f6c-32cfc5d1f07b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:29.338067 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e2d014-3ddc-42f2-8f6c-32cfc5d1f07b service cinder] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:29.349315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c822fd2-d11d-42fb-933a-d1b23e418fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:29.350740 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c822fd2-d11d-42fb-933a-d1b23e418fde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:29.351409 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c822fd2-d11d-42fb-933a-d1b23e418fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:29.351967 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c822fd2-d11d-42fb-933a-d1b23e418fde admin admin] PATH_INFO: `/v3/users/7816509ce2984ad2a134d87d8dacfa87` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:29.355271 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c822fd2-d11d-42fb-933a-d1b23e418fde admin admin] RBAC: Authorizing `identity:delete_user(user_id=7816509ce2984ad2a134d87d8dacfa87)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:29.358211 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c822fd2-d11d-42fb-933a-d1b23e418fde admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:29.423238 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-1c822fd2-d11d-42fb-933a-d1b23e418fde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7816509ce2984ad2a134d87d8dacfa87'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:29.435010 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-1c822fd2-d11d-42fb-933a-d1b23e418fde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7816509ce2984ad2a134d87d8dacfa87'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:29.453808 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-1c822fd2-d11d-42fb-933a-d1b23e418fde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7816509ce2984ad2a134d87d8dacfa87'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:29.454690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-1c822fd2-d11d-42fb-933a-d1b23e418fde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7816509ce2984ad2a134d87d8dacfa87'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:29.465643 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-1c822fd2-d11d-42fb-933a-d1b23e418fde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7816509ce2984ad2a134d87d8dacfa87'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:29.473858 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1139/2260] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:14:29 2022] DELETE /identity/v3/users/7816509ce2984ad2a134d87d8dacfa87 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:14:29.482965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a4acf-0793-4d5a-8df9-fc2ea3613dfc None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:29.484793 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a4acf-0793-4d5a-8df9-fc2ea3613dfc None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:29.486081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a4acf-0793-4d5a-8df9-fc2ea3613dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:29.524673 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-4e0a4acf-0793-4d5a-8df9-fc2ea3613dfc None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:14:29.528227 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-4e0a4acf-0793-4d5a-8df9-fc2ea3613dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:29.678629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1122/2261] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:14:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:14:29.693690 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c822fd2-d11d-42fb-933a-d1b23e418fde admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:29.696293 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-1c822fd2-d11d-42fb-933a-d1b23e418fde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:29.792783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c822fd2-d11d-42fb-933a-d1b23e418fde admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:29.798980 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d1ca1-52c7-48f6-85f6-642003ed6daf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:29.800141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d1ca1-52c7-48f6-85f6-642003ed6daf service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:29.800608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d1ca1-52c7-48f6-85f6-642003ed6daf service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:29.801100 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d1ca1-52c7-48f6-85f6-642003ed6daf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:29.817508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d1ca1-52c7-48f6-85f6-642003ed6daf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:29.819344 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d1ca1-52c7-48f6-85f6-642003ed6daf service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:29.833118 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1140/2262] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:14:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:14:30.247256 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a4acf-0793-4d5a-8df9-fc2ea3613dfc None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:30.250706 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-4e0a4acf-0793-4d5a-8df9-fc2ea3613dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:30.276305 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a4acf-0793-4d5a-8df9-fc2ea3613dfc None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:30.284955 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fdf200-ac29-48c5-9b30-ff0dc969d60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:30.289733 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fdf200-ac29-48c5-9b30-ff0dc969d60b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:30.290350 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fdf200-ac29-48c5-9b30-ff0dc969d60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:30.290728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fdf200-ac29-48c5-9b30-ff0dc969d60b admin admin] PATH_INFO: `/v3/projects/076251c420374fd4b9d325b4545b5db8` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:30.292364 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fdf200-ac29-48c5-9b30-ff0dc969d60b admin admin] RBAC: Authorizing `identity:delete_project(project_id=076251c420374fd4b9d325b4545b5db8)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:30.295268 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fdf200-ac29-48c5-9b30-ff0dc969d60b admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:30.347438 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:14:30.358707 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-17fdf200-ac29-48c5-9b30-ff0dc969d60b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '076251c420374fd4b9d325b4545b5db8', 'request_id': 'req-17fdf200-ac29-48c5-9b30-ff0dc969d60b'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:30.368920 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-17fdf200-ac29-48c5-9b30-ff0dc969d60b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '076251c420374fd4b9d325b4545b5db8', 'request_id': 'req-17fdf200-ac29-48c5-9b30-ff0dc969d60b'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:30.376605 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-17fdf200-ac29-48c5-9b30-ff0dc969d60b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '076251c420374fd4b9d325b4545b5db8', 'request_id': 'req-17fdf200-ac29-48c5-9b30-ff0dc969d60b'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:30.390908 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-17fdf200-ac29-48c5-9b30-ff0dc969d60b admin admin] The token cache is being invalidate because project 076251c420374fd4b9d325b4545b5db8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:14:30.391223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-17fdf200-ac29-48c5-9b30-ff0dc969d60b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:30.391885 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-17fdf200-ac29-48c5-9b30-ff0dc969d60b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:30.394437 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1123/2263] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:14:30 2022] DELETE /identity/v3/projects/076251c420374fd4b9d325b4545b5db8 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:14:39.827464 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=80,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:14:40.387174 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=10,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:14:41.196050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d82de-ad36-47ee-a186-0d40f8d34f1a None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:41.197211 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d82de-ad36-47ee-a186-0d40f8d34f1a None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:41.197557 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d82de-ad36-47ee-a186-0d40f8d34f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:41.230344 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-9c2d82de-ad36-47ee-a186-0d40f8d34f1a None None] MFA Rules not processed for user `c44105f2d90e469fb03f311a4cf1476f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:14:41.232441 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-9c2d82de-ad36-47ee-a186-0d40f8d34f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:41.334171 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1141/2264] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:14:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:14:45.981643 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fdf200-ac29-48c5-9b30-ff0dc969d60b admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:45.983336 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-17fdf200-ac29-48c5-9b30-ff0dc969d60b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:46.064799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fdf200-ac29-48c5-9b30-ff0dc969d60b admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:46.069558 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7928f-ca7c-4f29-844c-ced70b9d297b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:46.070206 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7928f-ca7c-4f29-844c-ced70b9d297b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:46.070480 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7928f-ca7c-4f29-844c-ced70b9d297b admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:46.070745 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7928f-ca7c-4f29-844c-ced70b9d297b admin admin] PATH_INFO: `/v3/users/9d20cc2558a94089a64fe8a3800b96a0` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:46.072298 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde7928f-ca7c-4f29-844c-ced70b9d297b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d20cc2558a94089a64fe8a3800b96a0)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:46.074396 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde7928f-ca7c-4f29-844c-ced70b9d297b admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:46.134234 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cde7928f-ca7c-4f29-844c-ced70b9d297b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d20cc2558a94089a64fe8a3800b96a0'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:46.143081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cde7928f-ca7c-4f29-844c-ced70b9d297b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d20cc2558a94089a64fe8a3800b96a0'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:46.157832 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cde7928f-ca7c-4f29-844c-ced70b9d297b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d20cc2558a94089a64fe8a3800b96a0'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:46.158750 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cde7928f-ca7c-4f29-844c-ced70b9d297b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d20cc2558a94089a64fe8a3800b96a0'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:46.173742 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cde7928f-ca7c-4f29-844c-ced70b9d297b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d20cc2558a94089a64fe8a3800b96a0'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:46.185348 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1124/2265] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:14:45 2022] DELETE /identity/v3/users/9d20cc2558a94089a64fe8a3800b96a0 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:14:46.192448 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ba842-d853-40be-a7ea-4caf3b7ff4eb None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:46.193267 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ba842-d853-40be-a7ea-4caf3b7ff4eb None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:46.193267 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ba842-d853-40be-a7ea-4caf3b7ff4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:46.217802 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-3a1ba842-d853-40be-a7ea-4caf3b7ff4eb None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:14:46.219361 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-3a1ba842-d853-40be-a7ea-4caf3b7ff4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:46.307955 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1142/2266] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:14:46.321291 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7928f-ca7c-4f29-844c-ced70b9d297b admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:46.323372 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-cde7928f-ca7c-4f29-844c-ced70b9d297b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:46.426509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7928f-ca7c-4f29-844c-ced70b9d297b admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:46.431083 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27c61d-f5a3-4282-8df0-aae09996cf16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:46.431924 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27c61d-f5a3-4282-8df0-aae09996cf16 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:46.432296 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27c61d-f5a3-4282-8df0-aae09996cf16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:46.432627 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27c61d-f5a3-4282-8df0-aae09996cf16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:46.444505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de27c61d-f5a3-4282-8df0-aae09996cf16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:46.446019 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de27c61d-f5a3-4282-8df0-aae09996cf16 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:46.465961 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1125/2267] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:14:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:14:46.999874 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ba842-d853-40be-a7ea-4caf3b7ff4eb None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:47.003298 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-3a1ba842-d853-40be-a7ea-4caf3b7ff4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:47.018639 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ba842-d853-40be-a7ea-4caf3b7ff4eb None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:47.024521 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b2824-16e1-43e2-a6e5-df637c7e6a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:47.025188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b2824-16e1-43e2-a6e5-df637c7e6a0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:47.025470 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b2824-16e1-43e2-a6e5-df637c7e6a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:47.025733 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b2824-16e1-43e2-a6e5-df637c7e6a0c admin admin] PATH_INFO: `/v3/projects/6ce565ef16a4439d8b3aad55a2c30758` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:47.028298 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b2824-16e1-43e2-a6e5-df637c7e6a0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ce565ef16a4439d8b3aad55a2c30758)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:47.031764 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b2824-16e1-43e2-a6e5-df637c7e6a0c admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:47.090365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-0e0b2824-16e1-43e2-a6e5-df637c7e6a0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ce565ef16a4439d8b3aad55a2c30758', 'request_id': 'req-0e0b2824-16e1-43e2-a6e5-df637c7e6a0c'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:47.102480 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-0e0b2824-16e1-43e2-a6e5-df637c7e6a0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ce565ef16a4439d8b3aad55a2c30758', 'request_id': 'req-0e0b2824-16e1-43e2-a6e5-df637c7e6a0c'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:47.110403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-0e0b2824-16e1-43e2-a6e5-df637c7e6a0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ce565ef16a4439d8b3aad55a2c30758', 'request_id': 'req-0e0b2824-16e1-43e2-a6e5-df637c7e6a0c'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:47.129645 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-0e0b2824-16e1-43e2-a6e5-df637c7e6a0c admin admin] The token cache is being invalidate because project 6ce565ef16a4439d8b3aad55a2c30758 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:14:47.129947 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-0e0b2824-16e1-43e2-a6e5-df637c7e6a0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:47.130612 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-0e0b2824-16e1-43e2-a6e5-df637c7e6a0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:14:47.132163 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1143/2268] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:14:46 2022] DELETE /identity/v3/projects/6ce565ef16a4439d8b3aad55a2c30758 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:14:54.677381 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b642f2-fdd3-444b-9a30-d188c0bde09a None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:54.678044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b642f2-fdd3-444b-9a30-d188c0bde09a None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:54.678044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b642f2-fdd3-444b-9a30-d188c0bde09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:54.700782 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-16b642f2-fdd3-444b-9a30-d188c0bde09a None None] MFA Rules not processed for user `2bd47bf9aa4142b1b7ec8d02e8661578`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:14:54.702285 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-16b642f2-fdd3-444b-9a30-d188c0bde09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:54.724170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:14:54.799312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1126/2269] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:14:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:14:56.401487 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c188a26-a8fc-4d08-bb8e-c65697315b7f None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:56.402404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c188a26-a8fc-4d08-bb8e-c65697315b7f None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:56.402404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c188a26-a8fc-4d08-bb8e-c65697315b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:56.430606 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-4c188a26-a8fc-4d08-bb8e-c65697315b7f None None] MFA Rules not processed for user `3fada78ceaeb4f3bad77c516f6dfcfd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:14:56.432143 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-4c188a26-a8fc-4d08-bb8e-c65697315b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:56.484361 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:14:56.552020 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1144/2270] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 15:14:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:14:56.630809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b642f2-fdd3-444b-9a30-d188c0bde09a None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:14:56.632383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-16b642f2-fdd3-444b-9a30-d188c0bde09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:14:56.731588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b642f2-fdd3-444b-9a30-d188c0bde09a None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:14:56.742573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f7970-40a1-4f9a-bbdb-a57276cf4621 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05aa5e40ee4542719657bc03b2015e72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:14:56.743569 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f7970-40a1-4f9a-bbdb-a57276cf4621 service swift] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:14:56.744046 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f7970-40a1-4f9a-bbdb-a57276cf4621 service swift] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:14:56.744511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f7970-40a1-4f9a-bbdb-a57276cf4621 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:14:56.761169 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95f7970-40a1-4f9a-bbdb-a57276cf4621 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:14:56.764195 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95f7970-40a1-4f9a-bbdb-a57276cf4621 service swift] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:14:56.780657 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1127/2271] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 15:14:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 15:15:01.170473 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c188a26-a8fc-4d08-bb8e-c65697315b7f None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:01.173194 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-4c188a26-a8fc-4d08-bb8e-c65697315b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:01.319238 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c188a26-a8fc-4d08-bb8e-c65697315b7f None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:01.325065 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b86e43-94fe-4cd8-9035-6fc1c9255071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:01.325946 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b86e43-94fe-4cd8-9035-6fc1c9255071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:01.326283 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b86e43-94fe-4cd8-9035-6fc1c9255071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:01.326750 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b86e43-94fe-4cd8-9035-6fc1c9255071 admin admin] PATH_INFO: `/v3/users/c6fdb6b74f574a54aff5b59032a61cf1` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:01.328449 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b86e43-94fe-4cd8-9035-6fc1c9255071 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6fdb6b74f574a54aff5b59032a61cf1)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:01.330545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b86e43-94fe-4cd8-9035-6fc1c9255071 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:01.438189 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-95b86e43-94fe-4cd8-9035-6fc1c9255071 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6fdb6b74f574a54aff5b59032a61cf1'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:01.463075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-95b86e43-94fe-4cd8-9035-6fc1c9255071 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6fdb6b74f574a54aff5b59032a61cf1'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:01.485300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-95b86e43-94fe-4cd8-9035-6fc1c9255071 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6fdb6b74f574a54aff5b59032a61cf1'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:01.486249 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-95b86e43-94fe-4cd8-9035-6fc1c9255071 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6fdb6b74f574a54aff5b59032a61cf1'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:01.509934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-95b86e43-94fe-4cd8-9035-6fc1c9255071 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6fdb6b74f574a54aff5b59032a61cf1'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:01.525198 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1145/2272] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:15:01 2022] DELETE /identity/v3/users/c6fdb6b74f574a54aff5b59032a61cf1 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:01.527664 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126aaf0-c590-4147-9925-fe1407b40693 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:01.528252 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126aaf0-c590-4147-9925-fe1407b40693 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:01.528252 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126aaf0-c590-4147-9925-fe1407b40693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:01.552399 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-f126aaf0-c590-4147-9925-fe1407b40693 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:01.554068 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f126aaf0-c590-4147-9925-fe1407b40693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:01.664440 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1128/2273] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:01.677159 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b86e43-94fe-4cd8-9035-6fc1c9255071 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:01.678671 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-95b86e43-94fe-4cd8-9035-6fc1c9255071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:01.802290 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b86e43-94fe-4cd8-9035-6fc1c9255071 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:01.806614 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f70f2-e1fd-49fc-8967-70cf33b7a932 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:01.807552 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f70f2-e1fd-49fc-8967-70cf33b7a932 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:01.807901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f70f2-e1fd-49fc-8967-70cf33b7a932 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:01.808333 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f70f2-e1fd-49fc-8967-70cf33b7a932 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:01.833335 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:01.837465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f70f2-e1fd-49fc-8967-70cf33b7a932 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:01.843896 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f70f2-e1fd-49fc-8967-70cf33b7a932 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:01.879677 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1146/2274] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:15:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:15:02.602854 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126aaf0-c590-4147-9925-fe1407b40693 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:02.607130 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f126aaf0-c590-4147-9925-fe1407b40693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:02.659563 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126aaf0-c590-4147-9925-fe1407b40693 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:02.666393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e615266-7e94-49f8-9650-156689642d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:02.667208 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e615266-7e94-49f8-9650-156689642d84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:02.667544 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e615266-7e94-49f8-9650-156689642d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:02.667884 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e615266-7e94-49f8-9650-156689642d84 admin admin] PATH_INFO: `/v3/projects/bd0ba18603f74d278d289ccf43959a18` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:02.669500 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e615266-7e94-49f8-9650-156689642d84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd0ba18603f74d278d289ccf43959a18)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:02.672037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e615266-7e94-49f8-9650-156689642d84 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:02.759297 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-2e615266-7e94-49f8-9650-156689642d84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd0ba18603f74d278d289ccf43959a18', 'request_id': 'req-2e615266-7e94-49f8-9650-156689642d84'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:02.766475 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:02.771915 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-2e615266-7e94-49f8-9650-156689642d84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd0ba18603f74d278d289ccf43959a18', 'request_id': 'req-2e615266-7e94-49f8-9650-156689642d84'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:02.786130 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-2e615266-7e94-49f8-9650-156689642d84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd0ba18603f74d278d289ccf43959a18', 'request_id': 'req-2e615266-7e94-49f8-9650-156689642d84'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:02.816805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-2e615266-7e94-49f8-9650-156689642d84 admin admin] The token cache is being invalidate because project bd0ba18603f74d278d289ccf43959a18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:15:02.817726 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-2e615266-7e94-49f8-9650-156689642d84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:02.818759 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-2e615266-7e94-49f8-9650-156689642d84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:02.827648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f70f2-e1fd-49fc-8967-70cf33b7a932 service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:02.835278 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1129/2275] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:15:02 2022] DELETE /identity/v3/projects/bd0ba18603f74d278d289ccf43959a18 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:02.845054 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f70f2-e1fd-49fc-8967-70cf33b7a932 service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:03.052357 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a2e9a-772e-4039-b8e5-18385b6681e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:03.053108 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a2e9a-772e-4039-b8e5-18385b6681e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:03.053654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a2e9a-772e-4039-b8e5-18385b6681e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:03.054161 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a2e9a-772e-4039-b8e5-18385b6681e0 admin admin] PATH_INFO: `/v3/users/522e80f4136849a9a7b1716e8cb36348` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:03.056139 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a2e9a-772e-4039-b8e5-18385b6681e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=522e80f4136849a9a7b1716e8cb36348)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:03.059956 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a2e9a-772e-4039-b8e5-18385b6681e0 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:03.215250 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2d0a2e9a-772e-4039-b8e5-18385b6681e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '522e80f4136849a9a7b1716e8cb36348'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:03.237647 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2d0a2e9a-772e-4039-b8e5-18385b6681e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '522e80f4136849a9a7b1716e8cb36348'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:03.283500 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2d0a2e9a-772e-4039-b8e5-18385b6681e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '522e80f4136849a9a7b1716e8cb36348'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:03.284643 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2d0a2e9a-772e-4039-b8e5-18385b6681e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '522e80f4136849a9a7b1716e8cb36348'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:03.304076 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2d0a2e9a-772e-4039-b8e5-18385b6681e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '522e80f4136849a9a7b1716e8cb36348'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:03.339311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1147/2276] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:15:02 2022] DELETE /identity/v3/users/522e80f4136849a9a7b1716e8cb36348 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:04.111305 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e615266-7e94-49f8-9650-156689642d84 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:04.112921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-2e615266-7e94-49f8-9650-156689642d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:04.129075 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e615266-7e94-49f8-9650-156689642d84 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:04.336820 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023d7b1-a31e-40af-afc3-5cc3a0c791aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:04.337207 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023d7b1-a31e-40af-afc3-5cc3a0c791aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:04.338929 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023d7b1-a31e-40af-afc3-5cc3a0c791aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:04.338929 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023d7b1-a31e-40af-afc3-5cc3a0c791aa admin admin] PATH_INFO: `/v3/projects/a20323e948dc40d5a8944cf72346fe54` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:04.340988 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8023d7b1-a31e-40af-afc3-5cc3a0c791aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=a20323e948dc40d5a8944cf72346fe54)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:04.344645 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8023d7b1-a31e-40af-afc3-5cc3a0c791aa admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:04.493383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8023d7b1-a31e-40af-afc3-5cc3a0c791aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a20323e948dc40d5a8944cf72346fe54', 'request_id': 'req-8023d7b1-a31e-40af-afc3-5cc3a0c791aa'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:04.512048 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8023d7b1-a31e-40af-afc3-5cc3a0c791aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a20323e948dc40d5a8944cf72346fe54', 'request_id': 'req-8023d7b1-a31e-40af-afc3-5cc3a0c791aa'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:04.533062 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8023d7b1-a31e-40af-afc3-5cc3a0c791aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a20323e948dc40d5a8944cf72346fe54', 'request_id': 'req-8023d7b1-a31e-40af-afc3-5cc3a0c791aa'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:04.568311 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8023d7b1-a31e-40af-afc3-5cc3a0c791aa admin admin] The token cache is being invalidate because project a20323e948dc40d5a8944cf72346fe54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:15:04.568943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8023d7b1-a31e-40af-afc3-5cc3a0c791aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:04.570175 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8023d7b1-a31e-40af-afc3-5cc3a0c791aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:04.573193 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1130/2277] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:15:04 2022] DELETE /identity/v3/projects/a20323e948dc40d5a8944cf72346fe54 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:04.580966 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3e3bb-c246-4845-9b4f-6289594af51d None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:04.581935 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3e3bb-c246-4845-9b4f-6289594af51d None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:04.582104 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3e3bb-c246-4845-9b4f-6289594af51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:04.620175 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-46b3e3bb-c246-4845-9b4f-6289594af51d None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:04.622406 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-46b3e3bb-c246-4845-9b4f-6289594af51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:04.801592 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1148/2278] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:04.826119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a408f-8e52-4bc1-b135-6556423cce08 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:04.826119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a408f-8e52-4bc1-b135-6556423cce08 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:04.826119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a408f-8e52-4bc1-b135-6556423cce08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:04.864292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-2d6a408f-8e52-4bc1-b135-6556423cce08 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:04.867635 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-2d6a408f-8e52-4bc1-b135-6556423cce08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:04.889188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1131/2279] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:04.895066 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3e3bb-c246-4845-9b4f-6289594af51d None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:04.917969 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3e3bb-c246-4845-9b4f-6289594af51d None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:04.927856 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b33524-eac3-45e3-a51b-dea1d3c36649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:04.929329 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b33524-eac3-45e3-a51b-dea1d3c36649 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:04.931068 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b33524-eac3-45e3-a51b-dea1d3c36649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:04.931669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b33524-eac3-45e3-a51b-dea1d3c36649 admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:04.945591 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b33524-eac3-45e3-a51b-dea1d3c36649 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:15:04.948540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b33524-eac3-45e3-a51b-dea1d3c36649 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:04.962635 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b33524-eac3-45e3-a51b-dea1d3c36649 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:05.059187 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1149/2280] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:15:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:15:05.075061 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a408f-8e52-4bc1-b135-6556423cce08 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:05.136137 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a408f-8e52-4bc1-b135-6556423cce08 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:05.162096 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f648734-abb4-48f1-aa66-1cb998b01289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:05.163658 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f648734-abb4-48f1-aa66-1cb998b01289 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:05.164444 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f648734-abb4-48f1-aa66-1cb998b01289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:05.165273 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f648734-abb4-48f1-aa66-1cb998b01289 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:05.166893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f648734-abb4-48f1-aa66-1cb998b01289 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:05.183161 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f648734-abb4-48f1-aa66-1cb998b01289 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:05.315374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b33524-eac3-45e3-a51b-dea1d3c36649 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:05.316885 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1132/2281] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:15:05 2022] POST /identity/v3/projects => generated 369 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:15:05.325204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:05.338830 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b33524-eac3-45e3-a51b-dea1d3c36649 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:05.475495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6bb1d-236f-4c43-8473-13c7359e933e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:05.481518 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6bb1d-236f-4c43-8473-13c7359e933e admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:05.482422 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6bb1d-236f-4c43-8473-13c7359e933e admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:05.483185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6bb1d-236f-4c43-8473-13c7359e933e admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:05.485642 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb6bb1d-236f-4c43-8473-13c7359e933e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:05.489860 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb6bb1d-236f-4c43-8473-13c7359e933e admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:05.535924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1150/2282] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:15:05 2022] POST /identity/v3/users => generated 285 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:15:05.536812 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f648734-abb4-48f1-aa66-1cb998b01289 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:05.549714 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f648734-abb4-48f1-aa66-1cb998b01289 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:05.568898 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec685775-6fc4-445a-be30-0a420928740f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:05.569560 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec685775-6fc4-445a-be30-0a420928740f admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:05.571382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec685775-6fc4-445a-be30-0a420928740f admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:05.572066 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec685775-6fc4-445a-be30-0a420928740f admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:05.572766 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec685775-6fc4-445a-be30-0a420928740f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:05.578664 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec685775-6fc4-445a-be30-0a420928740f admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:05.594241 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1133/2283] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:15:05 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:15:05.595818 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6bb1d-236f-4c43-8473-13c7359e933e admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:05.628464 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6bb1d-236f-4c43-8473-13c7359e933e admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:05.653613 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d7101-1454-4b0b-bafd-e28b61ee6497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:05.654593 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d7101-1454-4b0b-bafd-e28b61ee6497 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:05.654963 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d7101-1454-4b0b-bafd-e28b61ee6497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:05.655184 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d7101-1454-4b0b-bafd-e28b61ee6497 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:05.656043 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d7101-1454-4b0b-bafd-e28b61ee6497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:05.662161 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d7101-1454-4b0b-bafd-e28b61ee6497 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:05.754244 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec685775-6fc4-445a-be30-0a420928740f admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:05.761684 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1151/2284] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:15:05 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:15:05.775100 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec685775-6fc4-445a-be30-0a420928740f admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:05.813277 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d65fc-6cee-43eb-a1b3-6ea6d18caac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:05.819558 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d65fc-6cee-43eb-a1b3-6ea6d18caac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:05.819873 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d65fc-6cee-43eb-a1b3-6ea6d18caac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:05.820379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d65fc-6cee-43eb-a1b3-6ea6d18caac5 admin admin] PATH_INFO: `/v3/projects/b94f490c82af43e6bfe4816e56f3fa33/users/181672cb8f1e411385908a6aac5e4f8e/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:05.892559 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d65fc-6cee-43eb-a1b3-6ea6d18caac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b94f490c82af43e6bfe4816e56f3fa33, user_id=181672cb8f1e411385908a6aac5e4f8e, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:05.895821 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d65fc-6cee-43eb-a1b3-6ea6d18caac5 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:05.949089 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1134/2285] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:15:05 2022] PUT /identity/v3/projects/b94f490c82af43e6bfe4816e56f3fa33/users/181672cb8f1e411385908a6aac5e4f8e/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:05.970735 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6b3ee-95e7-4db8-8c01-e6743bb6c865 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:05.971557 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6b3ee-95e7-4db8-8c01-e6743bb6c865 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:05.972362 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6b3ee-95e7-4db8-8c01-e6743bb6c865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:06.028638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-f7c6b3ee-95e7-4db8-8c01-e6743bb6c865 None None] MFA Rules not processed for user `181672cb8f1e411385908a6aac5e4f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:06.030713 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-f7c6b3ee-95e7-4db8-8c01-e6743bb6c865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:06.270168 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d65fc-6cee-43eb-a1b3-6ea6d18caac5 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:06.286713 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1152/2286] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:06.294609 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:06.428889 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d65fc-6cee-43eb-a1b3-6ea6d18caac5 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:06.436059 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e582f-df45-433b-80d4-f931e3ee02a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:06.436712 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e582f-df45-433b-80d4-f931e3ee02a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:06.436993 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e582f-df45-433b-80d4-f931e3ee02a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:06.437310 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e582f-df45-433b-80d4-f931e3ee02a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:06.437974 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e582f-df45-433b-80d4-f931e3ee02a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:06.440583 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e582f-df45-433b-80d4-f931e3ee02a5 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:06.492251 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1135/2287] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:15:06 2022] POST /identity/v3/projects => generated 367 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:15:06.498280 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6b3ee-95e7-4db8-8c01-e6743bb6c865 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:06.535371 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6b3ee-95e7-4db8-8c01-e6743bb6c865 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:06.727637 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc3126-fbd9-4c6f-a8ad-d9e770934bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:06.728809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc3126-fbd9-4c6f-a8ad-d9e770934bab admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:06.728809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc3126-fbd9-4c6f-a8ad-d9e770934bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:06.728809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc3126-fbd9-4c6f-a8ad-d9e770934bab admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:06.729532 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dc3126-fbd9-4c6f-a8ad-d9e770934bab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:06.732899 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dc3126-fbd9-4c6f-a8ad-d9e770934bab admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:06.787473 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:06.797187 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1153/2288] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:15:06 2022] POST /identity/v3/users => generated 290 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:15:06.803597 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e582f-df45-433b-80d4-f931e3ee02a5 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:06.838255 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e582f-df45-433b-80d4-f931e3ee02a5 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:06.853203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f658649-6a09-4ad8-bd22-23dc8204a233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:06.856048 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f658649-6a09-4ad8-bd22-23dc8204a233 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:06.857780 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f658649-6a09-4ad8-bd22-23dc8204a233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:06.859753 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f658649-6a09-4ad8-bd22-23dc8204a233 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:06.861731 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f658649-6a09-4ad8-bd22-23dc8204a233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:06.866360 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f658649-6a09-4ad8-bd22-23dc8204a233 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:06.910018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1136/2289] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:15:06.916941 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc3126-fbd9-4c6f-a8ad-d9e770934bab admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:06.941753 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc3126-fbd9-4c6f-a8ad-d9e770934bab admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:06.957737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e0d02e-597b-44e1-acd7-31ba559e0b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:06.958933 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0d02e-597b-44e1-acd7-31ba559e0b33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:06.959411 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0d02e-597b-44e1-acd7-31ba559e0b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:06.959985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0d02e-597b-44e1-acd7-31ba559e0b33 admin admin] PATH_INFO: `/v3/projects/e9929154a59346988c377175409676a9/users/756416c5c9f1484baf4eede485f5fca1/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:07.007047 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e0d02e-597b-44e1-acd7-31ba559e0b33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9929154a59346988c377175409676a9, user_id=756416c5c9f1484baf4eede485f5fca1, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:07.010775 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e0d02e-597b-44e1-acd7-31ba559e0b33 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:07.073983 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1154/2290] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:15:06 2022] PUT /identity/v3/projects/e9929154a59346988c377175409676a9/users/756416c5c9f1484baf4eede485f5fca1/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:07.079611 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f658649-6a09-4ad8-bd22-23dc8204a233 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:07.093303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f658649-6a09-4ad8-bd22-23dc8204a233 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:07.246295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d92154-1a42-422e-9f18-6f7316a6c6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:07.246945 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d92154-1a42-422e-9f18-6f7316a6c6af admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:07.247218 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d92154-1a42-422e-9f18-6f7316a6c6af admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:07.248126 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d92154-1a42-422e-9f18-6f7316a6c6af admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:07.248863 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d92154-1a42-422e-9f18-6f7316a6c6af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:07.253570 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d92154-1a42-422e-9f18-6f7316a6c6af admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:07.283618 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1137/2291] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:15:07.285983 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e0d02e-597b-44e1-acd7-31ba559e0b33 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:07.311689 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e0d02e-597b-44e1-acd7-31ba559e0b33 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:07.327437 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60afaa4e-742b-455a-8a34-4f99318df7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:07.329601 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60afaa4e-742b-455a-8a34-4f99318df7b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:07.330077 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60afaa4e-742b-455a-8a34-4f99318df7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:07.330291 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60afaa4e-742b-455a-8a34-4f99318df7b9 admin admin] PATH_INFO: `/v3/projects/e9929154a59346988c377175409676a9/users/756416c5c9f1484baf4eede485f5fca1/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:07.338125 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60afaa4e-742b-455a-8a34-4f99318df7b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9929154a59346988c377175409676a9, user_id=756416c5c9f1484baf4eede485f5fca1, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:07.348929 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60afaa4e-742b-455a-8a34-4f99318df7b9 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:07.377188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1155/2292] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:15:07 2022] PUT /identity/v3/projects/e9929154a59346988c377175409676a9/users/756416c5c9f1484baf4eede485f5fca1/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:07.399319 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361750a-6412-4c45-a164-9b00ba6aa0d7 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:07.400482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361750a-6412-4c45-a164-9b00ba6aa0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:07.401794 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361750a-6412-4c45-a164-9b00ba6aa0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:07.450699 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-d361750a-6412-4c45-a164-9b00ba6aa0d7 None None] MFA Rules not processed for user `756416c5c9f1484baf4eede485f5fca1`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:07.452383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-d361750a-6412-4c45-a164-9b00ba6aa0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:07.570574 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1138/2293] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:07 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:07.586449 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60afaa4e-742b-455a-8a34-4f99318df7b9 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:07.589235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-60afaa4e-742b-455a-8a34-4f99318df7b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:07.736658 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60afaa4e-742b-455a-8a34-4f99318df7b9 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:07.744735 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db60737b-f2fc-4d85-8ecf-10a10eef7a0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:07.745845 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60737b-f2fc-4d85-8ecf-10a10eef7a0d service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:07.746295 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60737b-f2fc-4d85-8ecf-10a10eef7a0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:07.746755 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60737b-f2fc-4d85-8ecf-10a10eef7a0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:07.778140 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db60737b-f2fc-4d85-8ecf-10a10eef7a0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:07.778140 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db60737b-f2fc-4d85-8ecf-10a10eef7a0d service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:07.798034 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1156/2294] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:15:07 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:15:07.857738 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361750a-6412-4c45-a164-9b00ba6aa0d7 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:07.859428 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-d361750a-6412-4c45-a164-9b00ba6aa0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:07.867391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:07.875206 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361750a-6412-4c45-a164-9b00ba6aa0d7 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:07.882655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8235eb69-4cc8-44a6-b9dc-f75ee5e6e738 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:07.883390 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235eb69-4cc8-44a6-b9dc-f75ee5e6e738 service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:07.883659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235eb69-4cc8-44a6-b9dc-f75ee5e6e738 service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:07.883977 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235eb69-4cc8-44a6-b9dc-f75ee5e6e738 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:08.052512 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8235eb69-4cc8-44a6-b9dc-f75ee5e6e738 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:08.054187 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8235eb69-4cc8-44a6-b9dc-f75ee5e6e738 service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:08.086798 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1139/2295] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:15:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:15:10.718538 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db60737b-f2fc-4d85-8ecf-10a10eef7a0d service cinder] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:10.720031 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-db60737b-f2fc-4d85-8ecf-10a10eef7a0d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:10.908140 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db60737b-f2fc-4d85-8ecf-10a10eef7a0d service cinder] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:10.914535 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430caa0c-45c6-4a3c-97f5-f3bcf7069657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:10.915263 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430caa0c-45c6-4a3c-97f5-f3bcf7069657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:10.915558 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430caa0c-45c6-4a3c-97f5-f3bcf7069657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:10.915829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430caa0c-45c6-4a3c-97f5-f3bcf7069657 admin admin] PATH_INFO: `/v3/users/6c58cf4b3ecc4881ba1fe8ae014c8aa9` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:10.917114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430caa0c-45c6-4a3c-97f5-f3bcf7069657 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c58cf4b3ecc4881ba1fe8ae014c8aa9)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:10.919767 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430caa0c-45c6-4a3c-97f5-f3bcf7069657 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:10.989890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-430caa0c-45c6-4a3c-97f5-f3bcf7069657 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c58cf4b3ecc4881ba1fe8ae014c8aa9'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:11.001034 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-430caa0c-45c6-4a3c-97f5-f3bcf7069657 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c58cf4b3ecc4881ba1fe8ae014c8aa9'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:11.020365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:11.023840 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-430caa0c-45c6-4a3c-97f5-f3bcf7069657 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c58cf4b3ecc4881ba1fe8ae014c8aa9'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:11.024648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-430caa0c-45c6-4a3c-97f5-f3bcf7069657 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c58cf4b3ecc4881ba1fe8ae014c8aa9'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:11.041126 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-430caa0c-45c6-4a3c-97f5-f3bcf7069657 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c58cf4b3ecc4881ba1fe8ae014c8aa9'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:11.054885 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1157/2296] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:15:10 2022] DELETE /identity/v3/users/6c58cf4b3ecc4881ba1fe8ae014c8aa9 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:11.059889 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded07e2-8b4a-4cf8-bea9-5614870868ee None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:11.060876 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded07e2-8b4a-4cf8-bea9-5614870868ee None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:11.060876 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded07e2-8b4a-4cf8-bea9-5614870868ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:11.084648 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-dded07e2-8b4a-4cf8-bea9-5614870868ee None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:11.086159 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-dded07e2-8b4a-4cf8-bea9-5614870868ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:11.193022 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1140/2297] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:11.207799 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430caa0c-45c6-4a3c-97f5-f3bcf7069657 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:11.210013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-430caa0c-45c6-4a3c-97f5-f3bcf7069657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:11.311718 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430caa0c-45c6-4a3c-97f5-f3bcf7069657 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:11.317355 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0820d00-e316-4788-9a38-462327a78342 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:11.318173 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0820d00-e316-4788-9a38-462327a78342 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:11.318465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0820d00-e316-4788-9a38-462327a78342 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:11.318715 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0820d00-e316-4788-9a38-462327a78342 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:11.337553 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0820d00-e316-4788-9a38-462327a78342 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:11.339816 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0820d00-e316-4788-9a38-462327a78342 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:11.353068 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1158/2298] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:15:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:15:11.980926 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded07e2-8b4a-4cf8-bea9-5614870868ee None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:12.017553 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-dded07e2-8b4a-4cf8-bea9-5614870868ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:12.057814 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded07e2-8b4a-4cf8-bea9-5614870868ee None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:12.088488 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698dbef-3499-4e82-a36c-3be5ea56f3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:12.089207 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698dbef-3499-4e82-a36c-3be5ea56f3c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:12.089423 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698dbef-3499-4e82-a36c-3be5ea56f3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:12.089594 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698dbef-3499-4e82-a36c-3be5ea56f3c9 admin admin] PATH_INFO: `/v3/projects/dc1eb61863704af88224b447d96abd52` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:12.094950 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698dbef-3499-4e82-a36c-3be5ea56f3c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc1eb61863704af88224b447d96abd52)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:12.097192 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698dbef-3499-4e82-a36c-3be5ea56f3c9 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:12.261890 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8698dbef-3499-4e82-a36c-3be5ea56f3c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc1eb61863704af88224b447d96abd52', 'request_id': 'req-8698dbef-3499-4e82-a36c-3be5ea56f3c9'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:12.292180 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8698dbef-3499-4e82-a36c-3be5ea56f3c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc1eb61863704af88224b447d96abd52', 'request_id': 'req-8698dbef-3499-4e82-a36c-3be5ea56f3c9'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:12.299448 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8698dbef-3499-4e82-a36c-3be5ea56f3c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc1eb61863704af88224b447d96abd52', 'request_id': 'req-8698dbef-3499-4e82-a36c-3be5ea56f3c9'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:12.318147 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8698dbef-3499-4e82-a36c-3be5ea56f3c9 admin admin] The token cache is being invalidate because project dc1eb61863704af88224b447d96abd52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:15:12.318589 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8698dbef-3499-4e82-a36c-3be5ea56f3c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:12.319413 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8698dbef-3499-4e82-a36c-3be5ea56f3c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:12.324724 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1141/2299] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:15:11 2022] DELETE /identity/v3/projects/dc1eb61863704af88224b447d96abd52 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:12.327879 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e4910-3eea-4244-9cfc-7167fe22c742 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:12.328399 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e4910-3eea-4244-9cfc-7167fe22c742 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:12.328499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e4910-3eea-4244-9cfc-7167fe22c742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:12.350924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-db4e4910-3eea-4244-9cfc-7167fe22c742 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:12.352351 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-db4e4910-3eea-4244-9cfc-7167fe22c742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:12.461627 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1159/2300] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:15:12.469361 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69aad0f-9298-4061-9e91-a216cb74e113 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:12.469769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69aad0f-9298-4061-9e91-a216cb74e113 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:12.470190 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69aad0f-9298-4061-9e91-a216cb74e113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:12.498471 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-b69aad0f-9298-4061-9e91-a216cb74e113 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:12.500038 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b69aad0f-9298-4061-9e91-a216cb74e113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:12.510490 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1142/2301] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:12.517048 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e4910-3eea-4244-9cfc-7167fe22c742 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:12.530830 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e4910-3eea-4244-9cfc-7167fe22c742 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:12.539113 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd3cee-743e-44b5-b387-701fc5582fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:12.539758 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd3cee-743e-44b5-b387-701fc5582fe9 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:12.540329 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd3cee-743e-44b5-b387-701fc5582fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:12.540606 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd3cee-743e-44b5-b387-701fc5582fe9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:12.541303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd3cee-743e-44b5-b387-701fc5582fe9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:15:12.541607 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd3cee-743e-44b5-b387-701fc5582fe9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:12.543250 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd3cee-743e-44b5-b387-701fc5582fe9 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:12.563725 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1160/2302] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:15:12.568920 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69aad0f-9298-4061-9e91-a216cb74e113 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:12.585080 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69aad0f-9298-4061-9e91-a216cb74e113 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:12.592793 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31862d01-b651-499e-b874-d45c2dcc4d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:12.593933 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31862d01-b651-499e-b874-d45c2dcc4d7a admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:12.594380 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31862d01-b651-499e-b874-d45c2dcc4d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:12.594823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31862d01-b651-499e-b874-d45c2dcc4d7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:12.595690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31862d01-b651-499e-b874-d45c2dcc4d7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:12.599147 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31862d01-b651-499e-b874-d45c2dcc4d7a admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:12.622226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:12.638859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1143/2303] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:15:12 2022] POST /identity/v3/projects => generated 381 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:15:12.643700 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd3cee-743e-44b5-b387-701fc5582fe9 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:12.655724 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd3cee-743e-44b5-b387-701fc5582fe9 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:12.739917 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db02418-505f-4543-8058-d264222812af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:12.740548 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db02418-505f-4543-8058-d264222812af admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:12.741031 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db02418-505f-4543-8058-d264222812af admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:12.741385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db02418-505f-4543-8058-d264222812af admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:12.741998 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db02418-505f-4543-8058-d264222812af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:12.744025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db02418-505f-4543-8058-d264222812af admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:12.771040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1161/2304] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:15:12 2022] POST /identity/v3/users => generated 291 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:15:12.776083 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31862d01-b651-499e-b874-d45c2dcc4d7a admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:12.787959 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31862d01-b651-499e-b874-d45c2dcc4d7a admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:12.803807 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d087b8-3fa8-4b83-a19c-959faa2c7c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:12.804394 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d087b8-3fa8-4b83-a19c-959faa2c7c61 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:12.804671 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d087b8-3fa8-4b83-a19c-959faa2c7c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:12.804938 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d087b8-3fa8-4b83-a19c-959faa2c7c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:12.805709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d087b8-3fa8-4b83-a19c-959faa2c7c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:12.815187 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d087b8-3fa8-4b83-a19c-959faa2c7c61 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:12.828378 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1144/2305] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:15:12.832648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db02418-505f-4543-8058-d264222812af admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:12.840413 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:12.848569 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db02418-505f-4543-8058-d264222812af admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:12.882248 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ca6d2-06df-415d-ac3f-5cba36a7aab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:12.883005 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ca6d2-06df-415d-ac3f-5cba36a7aab8 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:12.883491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ca6d2-06df-415d-ac3f-5cba36a7aab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:12.883976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ca6d2-06df-415d-ac3f-5cba36a7aab8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:12.885199 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ca6d2-06df-415d-ac3f-5cba36a7aab8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:12.888320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ca6d2-06df-415d-ac3f-5cba36a7aab8 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:12.924051 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1162/2306] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:15:12.926514 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d087b8-3fa8-4b83-a19c-959faa2c7c61 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:12.948799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d087b8-3fa8-4b83-a19c-959faa2c7c61 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:12.962960 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23441df-9802-4dc4-ba81-4c526a7d711f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:12.964212 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23441df-9802-4dc4-ba81-4c526a7d711f admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:12.965032 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23441df-9802-4dc4-ba81-4c526a7d711f admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:12.965816 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23441df-9802-4dc4-ba81-4c526a7d711f admin admin] PATH_INFO: `/v3/projects/fd7984f5434f42098fb127572b6d1e0c/users/0292da3873ef419097717fd11337a128/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:12.990205 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23441df-9802-4dc4-ba81-4c526a7d711f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd7984f5434f42098fb127572b6d1e0c, user_id=0292da3873ef419097717fd11337a128, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:12.994027 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23441df-9802-4dc4-ba81-4c526a7d711f admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:13.013246 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1145/2307] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:15:12 2022] PUT /identity/v3/projects/fd7984f5434f42098fb127572b6d1e0c/users/0292da3873ef419097717fd11337a128/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:13.017089 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aea385-0df2-4f2f-b7cf-23ece4739675 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:13.017497 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aea385-0df2-4f2f-b7cf-23ece4739675 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:13.017744 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aea385-0df2-4f2f-b7cf-23ece4739675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:13.040456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-46aea385-0df2-4f2f-b7cf-23ece4739675 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:13.041755 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-46aea385-0df2-4f2f-b7cf-23ece4739675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:13.082704 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23441df-9802-4dc4-ba81-4c526a7d711f admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:13.084551 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b23441df-9802-4dc4-ba81-4c526a7d711f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:13.142592 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1163/2308] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:15:13.159202 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aea385-0df2-4f2f-b7cf-23ece4739675 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:13.161460 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-46aea385-0df2-4f2f-b7cf-23ece4739675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:13.179214 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23441df-9802-4dc4-ba81-4c526a7d711f admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:13.185964 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a83eb-f85b-4f1e-8b5a-37fcafe4f8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:13.186759 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a83eb-f85b-4f1e-8b5a-37fcafe4f8db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:13.187221 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a83eb-f85b-4f1e-8b5a-37fcafe4f8db admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:13.187527 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a83eb-f85b-4f1e-8b5a-37fcafe4f8db admin admin] PATH_INFO: `/v3/users/181672cb8f1e411385908a6aac5e4f8e` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:13.188924 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a83eb-f85b-4f1e-8b5a-37fcafe4f8db admin admin] RBAC: Authorizing `identity:delete_user(user_id=181672cb8f1e411385908a6aac5e4f8e)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:13.190977 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a83eb-f85b-4f1e-8b5a-37fcafe4f8db admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:13.270461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-4c1a83eb-f85b-4f1e-8b5a-37fcafe4f8db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '181672cb8f1e411385908a6aac5e4f8e'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:13.280866 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-4c1a83eb-f85b-4f1e-8b5a-37fcafe4f8db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '181672cb8f1e411385908a6aac5e4f8e'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:13.281166 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aea385-0df2-4f2f-b7cf-23ece4739675 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:13.286834 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3bcb08-fa53-4d96-8b3a-fa75a3e0f053 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:13.288648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bcb08-fa53-4d96-8b3a-fa75a3e0f053 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:13.288996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bcb08-fa53-4d96-8b3a-fa75a3e0f053 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:13.289329 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bcb08-fa53-4d96-8b3a-fa75a3e0f053 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:13.302079 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-4c1a83eb-f85b-4f1e-8b5a-37fcafe4f8db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '181672cb8f1e411385908a6aac5e4f8e'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:13.303271 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-4c1a83eb-f85b-4f1e-8b5a-37fcafe4f8db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '181672cb8f1e411385908a6aac5e4f8e'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:13.306571 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3bcb08-fa53-4d96-8b3a-fa75a3e0f053 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:13.308929 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3bcb08-fa53-4d96-8b3a-fa75a3e0f053 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:13.314526 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-4c1a83eb-f85b-4f1e-8b5a-37fcafe4f8db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '181672cb8f1e411385908a6aac5e4f8e'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:13.322578 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1146/2309] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:15:13 2022] DELETE /identity/v3/users/181672cb8f1e411385908a6aac5e4f8e => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:15:13.325365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1164/2310] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:15:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:15:13.331039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ee78b-b465-4131-9687-081f6d9c4f94 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:13.331322 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ee78b-b465-4131-9687-081f6d9c4f94 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:13.331863 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ee78b-b465-4131-9687-081f6d9c4f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:13.357581 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-8e3ee78b-b465-4131-9687-081f6d9c4f94 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:13.358844 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-8e3ee78b-b465-4131-9687-081f6d9c4f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:13.403902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:13.440861 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1147/2311] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:13.456606 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3bcb08-fa53-4d96-8b3a-fa75a3e0f053 service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:13.458845 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-bb3bcb08-fa53-4d96-8b3a-fa75a3e0f053 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:13.472452 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3bcb08-fa53-4d96-8b3a-fa75a3e0f053 service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:13.546710 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795994e-dfeb-4a2f-8683-e3a0f18e889a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:13.547795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795994e-dfeb-4a2f-8683-e3a0f18e889a service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:13.548283 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795994e-dfeb-4a2f-8683-e3a0f18e889a service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:13.548745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795994e-dfeb-4a2f-8683-e3a0f18e889a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:13.565693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4795994e-dfeb-4a2f-8683-e3a0f18e889a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:13.568458 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4795994e-dfeb-4a2f-8683-e3a0f18e889a service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:13.587768 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1165/2312] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:15:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:15:14.100609 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ee78b-b465-4131-9687-081f6d9c4f94 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:14.102818 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-8e3ee78b-b465-4131-9687-081f6d9c4f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:14.125715 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ee78b-b465-4131-9687-081f6d9c4f94 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:14.151697 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10fc0e-30f1-4eee-b0c7-740ba84b192e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:14.152549 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10fc0e-30f1-4eee-b0c7-740ba84b192e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:14.152862 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10fc0e-30f1-4eee-b0c7-740ba84b192e admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:14.153475 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10fc0e-30f1-4eee-b0c7-740ba84b192e admin admin] PATH_INFO: `/v3/projects/b94f490c82af43e6bfe4816e56f3fa33` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:14.155274 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10fc0e-30f1-4eee-b0c7-740ba84b192e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b94f490c82af43e6bfe4816e56f3fa33)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:14.158440 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10fc0e-30f1-4eee-b0c7-740ba84b192e admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:14.220452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-6e10fc0e-30f1-4eee-b0c7-740ba84b192e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b94f490c82af43e6bfe4816e56f3fa33', 'request_id': 'req-6e10fc0e-30f1-4eee-b0c7-740ba84b192e'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:14.230045 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-6e10fc0e-30f1-4eee-b0c7-740ba84b192e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b94f490c82af43e6bfe4816e56f3fa33', 'request_id': 'req-6e10fc0e-30f1-4eee-b0c7-740ba84b192e'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:14.237082 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-6e10fc0e-30f1-4eee-b0c7-740ba84b192e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b94f490c82af43e6bfe4816e56f3fa33', 'request_id': 'req-6e10fc0e-30f1-4eee-b0c7-740ba84b192e'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:14.250157 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-6e10fc0e-30f1-4eee-b0c7-740ba84b192e admin admin] The token cache is being invalidate because project b94f490c82af43e6bfe4816e56f3fa33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:15:14.250448 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-6e10fc0e-30f1-4eee-b0c7-740ba84b192e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:14.251116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-6e10fc0e-30f1-4eee-b0c7-740ba84b192e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:14.252698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1148/2313] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:15:14 2022] DELETE /identity/v3/projects/b94f490c82af43e6bfe4816e56f3fa33 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:14.257583 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795994e-dfeb-4a2f-8683-e3a0f18e889a service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:14.272059 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795994e-dfeb-4a2f-8683-e3a0f18e889a service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:14.326844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:14.363443 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f89ff-3dcd-4e6b-8f23-27b3a537a6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:14.364683 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f89ff-3dcd-4e6b-8f23-27b3a537a6a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:14.365349 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f89ff-3dcd-4e6b-8f23-27b3a537a6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:14.366502 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f89ff-3dcd-4e6b-8f23-27b3a537a6a6 admin admin] PATH_INFO: `/v3/users/756416c5c9f1484baf4eede485f5fca1` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:14.368385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f89ff-3dcd-4e6b-8f23-27b3a537a6a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=756416c5c9f1484baf4eede485f5fca1)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:14.371928 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f89ff-3dcd-4e6b-8f23-27b3a537a6a6 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:14.517701 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-bd8f89ff-3dcd-4e6b-8f23-27b3a537a6a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '756416c5c9f1484baf4eede485f5fca1'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:14.540276 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-bd8f89ff-3dcd-4e6b-8f23-27b3a537a6a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '756416c5c9f1484baf4eede485f5fca1'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:14.560635 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-bd8f89ff-3dcd-4e6b-8f23-27b3a537a6a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '756416c5c9f1484baf4eede485f5fca1'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:14.561587 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-bd8f89ff-3dcd-4e6b-8f23-27b3a537a6a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '756416c5c9f1484baf4eede485f5fca1'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:14.576659 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-bd8f89ff-3dcd-4e6b-8f23-27b3a537a6a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '756416c5c9f1484baf4eede485f5fca1'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:14.585546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1166/2314] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:15:14 2022] DELETE /identity/v3/users/756416c5c9f1484baf4eede485f5fca1 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:14.978564 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14af48c-a453-4716-b3ac-130518e71cf6 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:14.978983 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14af48c-a453-4716-b3ac-130518e71cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:14.979600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14af48c-a453-4716-b3ac-130518e71cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:15.042356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-a14af48c-a453-4716-b3ac-130518e71cf6 None None] MFA Rules not processed for user `0292da3873ef419097717fd11337a128`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:15.046292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-a14af48c-a453-4716-b3ac-130518e71cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:15.143040 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1149/2315] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:15.155684 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f89ff-3dcd-4e6b-8f23-27b3a537a6a6 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:15.157252 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-bd8f89ff-3dcd-4e6b-8f23-27b3a537a6a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:15.240497 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f89ff-3dcd-4e6b-8f23-27b3a537a6a6 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:15.246760 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675001d-3d3f-4644-9670-00faabd4ed6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:15.247678 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675001d-3d3f-4644-9670-00faabd4ed6d service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:15.253189 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675001d-3d3f-4644-9670-00faabd4ed6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:15.253189 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675001d-3d3f-4644-9670-00faabd4ed6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:15.258986 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5675001d-3d3f-4644-9670-00faabd4ed6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:15.260632 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5675001d-3d3f-4644-9670-00faabd4ed6d service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:15.276511 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1167/2316] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:15:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:15:15.366231 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14af48c-a453-4716-b3ac-130518e71cf6 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:15.368639 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-a14af48c-a453-4716-b3ac-130518e71cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:15.432985 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:15.474981 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14af48c-a453-4716-b3ac-130518e71cf6 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:15.488313 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ab7f6-f8ab-477d-8647-1661d61c9ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:15.489672 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ab7f6-f8ab-477d-8647-1661d61c9ce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:15.490121 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ab7f6-f8ab-477d-8647-1661d61c9ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:15.490990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ab7f6-f8ab-477d-8647-1661d61c9ce8 admin admin] PATH_INFO: `/v3/projects/e9929154a59346988c377175409676a9` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:15.494024 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ab7f6-f8ab-477d-8647-1661d61c9ce8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9929154a59346988c377175409676a9)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:15.499378 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ab7f6-f8ab-477d-8647-1661d61c9ce8 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:15.612233 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-fe6ab7f6-f8ab-477d-8647-1661d61c9ce8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9929154a59346988c377175409676a9', 'request_id': 'req-fe6ab7f6-f8ab-477d-8647-1661d61c9ce8'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:15.629440 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-fe6ab7f6-f8ab-477d-8647-1661d61c9ce8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9929154a59346988c377175409676a9', 'request_id': 'req-fe6ab7f6-f8ab-477d-8647-1661d61c9ce8'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:15.648137 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-fe6ab7f6-f8ab-477d-8647-1661d61c9ce8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9929154a59346988c377175409676a9', 'request_id': 'req-fe6ab7f6-f8ab-477d-8647-1661d61c9ce8'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:15.686708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-fe6ab7f6-f8ab-477d-8647-1661d61c9ce8 admin admin] The token cache is being invalidate because project e9929154a59346988c377175409676a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:15:15.687277 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-fe6ab7f6-f8ab-477d-8647-1661d61c9ce8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:15.689054 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-fe6ab7f6-f8ab-477d-8647-1661d61c9ce8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:15.692485 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1150/2317] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:15:15 2022] DELETE /identity/v3/projects/e9929154a59346988c377175409676a9 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:15.699106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f546d-6f9f-4339-a71c-b0f5443edd95 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:15.700069 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f546d-6f9f-4339-a71c-b0f5443edd95 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:15.701377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f546d-6f9f-4339-a71c-b0f5443edd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:15.750877 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-480f546d-6f9f-4339-a71c-b0f5443edd95 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:15.754842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-480f546d-6f9f-4339-a71c-b0f5443edd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:15.900648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1168/2318] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:15:15.910567 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b3a92-5c21-43ec-ab37-50bec8ce8110 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:15.911619 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b3a92-5c21-43ec-ab37-50bec8ce8110 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:15.912506 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b3a92-5c21-43ec-ab37-50bec8ce8110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:15.966018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-2a5b3a92-5c21-43ec-ab37-50bec8ce8110 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:15.968864 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-2a5b3a92-5c21-43ec-ab37-50bec8ce8110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:15.987723 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1151/2319] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:15.991872 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f546d-6f9f-4339-a71c-b0f5443edd95 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:16.002225 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:16.013979 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f546d-6f9f-4339-a71c-b0f5443edd95 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:16.024848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1300ce-6b4c-451f-bd6e-2c7de98b0c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:16.026883 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1300ce-6b4c-451f-bd6e-2c7de98b0c54 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:16.027521 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1300ce-6b4c-451f-bd6e-2c7de98b0c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:16.027699 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1300ce-6b4c-451f-bd6e-2c7de98b0c54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:16.028745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1300ce-6b4c-451f-bd6e-2c7de98b0c54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:15:16.029165 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1300ce-6b4c-451f-bd6e-2c7de98b0c54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:16.032166 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1300ce-6b4c-451f-bd6e-2c7de98b0c54 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:16.050304 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1169/2320] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:15:16.055068 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b3a92-5c21-43ec-ab37-50bec8ce8110 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:16.069747 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b3a92-5c21-43ec-ab37-50bec8ce8110 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:16.084248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72570289-d10f-425b-8dbc-d7caea83e782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:16.085410 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72570289-d10f-425b-8dbc-d7caea83e782 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:16.086099 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72570289-d10f-425b-8dbc-d7caea83e782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:16.086570 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72570289-d10f-425b-8dbc-d7caea83e782 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:16.089926 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72570289-d10f-425b-8dbc-d7caea83e782 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:16.093221 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72570289-d10f-425b-8dbc-d7caea83e782 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:16.140980 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1152/2321] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:15:16 2022] POST /identity/v3/projects => generated 383 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:15:16.145201 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1300ce-6b4c-451f-bd6e-2c7de98b0c54 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:16.159263 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1300ce-6b4c-451f-bd6e-2c7de98b0c54 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:16.276770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28793b-4ab0-47e7-86e2-bfb248a4a048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:16.279105 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28793b-4ab0-47e7-86e2-bfb248a4a048 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:16.279483 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28793b-4ab0-47e7-86e2-bfb248a4a048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:16.279636 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28793b-4ab0-47e7-86e2-bfb248a4a048 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:16.280275 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa28793b-4ab0-47e7-86e2-bfb248a4a048 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:16.282306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa28793b-4ab0-47e7-86e2-bfb248a4a048 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:16.317750 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1170/2322] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:15:16 2022] POST /identity/v3/users => generated 292 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:15:16.319504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72570289-d10f-425b-8dbc-d7caea83e782 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:16.343421 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72570289-d10f-425b-8dbc-d7caea83e782 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:16.368871 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34afb9-ccd8-4bc8-bc61-68a6f779836b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:16.369841 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34afb9-ccd8-4bc8-bc61-68a6f779836b admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:16.369841 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34afb9-ccd8-4bc8-bc61-68a6f779836b admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:16.370205 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34afb9-ccd8-4bc8-bc61-68a6f779836b admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:16.370824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e34afb9-ccd8-4bc8-bc61-68a6f779836b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:16.372530 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e34afb9-ccd8-4bc8-bc61-68a6f779836b admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:16.387884 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1153/2323] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:15:16.394635 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28793b-4ab0-47e7-86e2-bfb248a4a048 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:16.413510 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28793b-4ab0-47e7-86e2-bfb248a4a048 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:16.420271 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2b9e1-5e45-49b8-b2f2-55c7ce45f648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:16.421175 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2b9e1-5e45-49b8-b2f2-55c7ce45f648 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:16.421899 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2b9e1-5e45-49b8-b2f2-55c7ce45f648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:16.422092 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2b9e1-5e45-49b8-b2f2-55c7ce45f648 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:16.423905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2b9e1-5e45-49b8-b2f2-55c7ce45f648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:16.426201 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2b9e1-5e45-49b8-b2f2-55c7ce45f648 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:16.448451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1171/2324] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:15:16.452910 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34afb9-ccd8-4bc8-bc61-68a6f779836b admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:16.463491 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34afb9-ccd8-4bc8-bc61-68a6f779836b admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:16.469021 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb856da-4504-4a07-a721-adb2f471da93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:16.469671 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb856da-4504-4a07-a721-adb2f471da93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:16.469956 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb856da-4504-4a07-a721-adb2f471da93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:16.470216 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb856da-4504-4a07-a721-adb2f471da93 admin admin] PATH_INFO: `/v3/projects/fafd2f9bb2f34b20b7280784c9f065a9/users/bfa65f323c314aa19332f81d190939d8/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:16.493501 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb856da-4504-4a07-a721-adb2f471da93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fafd2f9bb2f34b20b7280784c9f065a9, user_id=bfa65f323c314aa19332f81d190939d8, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:16.496321 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb856da-4504-4a07-a721-adb2f471da93 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:16.517169 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1154/2325] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:15:16 2022] PUT /identity/v3/projects/fafd2f9bb2f34b20b7280784c9f065a9/users/bfa65f323c314aa19332f81d190939d8/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:16.528215 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b2a4d-c216-4831-9c80-5ce8fc034fcd None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:16.529877 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b2a4d-c216-4831-9c80-5ce8fc034fcd None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:16.530435 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b2a4d-c216-4831-9c80-5ce8fc034fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:16.577352 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-8f6b2a4d-c216-4831-9c80-5ce8fc034fcd None None] MFA Rules not processed for user `bfa65f323c314aa19332f81d190939d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:16.578886 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-8f6b2a4d-c216-4831-9c80-5ce8fc034fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:16.726146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1172/2326] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:16.730916 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb856da-4504-4a07-a721-adb2f471da93 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:16.948474 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb856da-4504-4a07-a721-adb2f471da93 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:16.956631 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34258ebe-00ae-4650-b3ae-092f1e2bf2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:16.958169 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34258ebe-00ae-4650-b3ae-092f1e2bf2eb admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:16.958552 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34258ebe-00ae-4650-b3ae-092f1e2bf2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:16.958923 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34258ebe-00ae-4650-b3ae-092f1e2bf2eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:16.959557 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba407e80-45bc-4026-8f14-706786e98513 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:16.959938 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba407e80-45bc-4026-8f14-706786e98513 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:16.960112 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34258ebe-00ae-4650-b3ae-092f1e2bf2eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:16.960429 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba407e80-45bc-4026-8f14-706786e98513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:16.962706 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34258ebe-00ae-4650-b3ae-092f1e2bf2eb admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:17.008122 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1155/2327] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:15:16 2022] POST /identity/v3/projects => generated 383 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:15:17.010758 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34258ebe-00ae-4650-b3ae-092f1e2bf2eb admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:17.011726 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-ba407e80-45bc-4026-8f14-706786e98513 None None] MFA Rules not processed for user `0292da3873ef419097717fd11337a128`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:17.013926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-ba407e80-45bc-4026-8f14-706786e98513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:17.023305 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:17.119507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34258ebe-00ae-4650-b3ae-092f1e2bf2eb admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:17.126690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680014fe-0351-45a9-8739-26bd1500e22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:17.127477 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680014fe-0351-45a9-8739-26bd1500e22d admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:17.127940 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680014fe-0351-45a9-8739-26bd1500e22d admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:17.128303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680014fe-0351-45a9-8739-26bd1500e22d admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:17.129058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680014fe-0351-45a9-8739-26bd1500e22d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:17.133553 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680014fe-0351-45a9-8739-26bd1500e22d admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:17.138745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1173/2328] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:17.168966 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1156/2329] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:15:17 2022] POST /identity/v3/users => generated 298 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:15:17.169574 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba407e80-45bc-4026-8f14-706786e98513 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:17.180044 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:17.201410 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba407e80-45bc-4026-8f14-706786e98513 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:17.208435 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd84d3f-632f-47d4-bdca-6ecf6d5fa941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:17.209259 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd84d3f-632f-47d4-bdca-6ecf6d5fa941 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:17.209641 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd84d3f-632f-47d4-bdca-6ecf6d5fa941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:17.210008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd84d3f-632f-47d4-bdca-6ecf6d5fa941 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:17.210810 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd84d3f-632f-47d4-bdca-6ecf6d5fa941 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:17.213024 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd84d3f-632f-47d4-bdca-6ecf6d5fa941 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:17.231262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1174/2330] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:15:17.232211 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680014fe-0351-45a9-8739-26bd1500e22d admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:17.246287 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680014fe-0351-45a9-8739-26bd1500e22d admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:17.256613 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96f565-e488-4b74-989d-6869d86ffecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:17.257875 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96f565-e488-4b74-989d-6869d86ffecc admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:17.266500 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96f565-e488-4b74-989d-6869d86ffecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:17.267103 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96f565-e488-4b74-989d-6869d86ffecc admin admin] PATH_INFO: `/v3/projects/83fe15d16b0c423cb1c440dae6d13ffd/users/190915f69ee04745845ae6dc52892755/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:17.310335 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96f565-e488-4b74-989d-6869d86ffecc admin admin] RBAC: Authorizing `identity:create_grant(project_id=83fe15d16b0c423cb1c440dae6d13ffd, user_id=190915f69ee04745845ae6dc52892755, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:17.316159 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96f565-e488-4b74-989d-6869d86ffecc admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:17.350647 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1157/2331] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:15:17 2022] PUT /identity/v3/projects/83fe15d16b0c423cb1c440dae6d13ffd/users/190915f69ee04745845ae6dc52892755/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:15:17.358291 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd84d3f-632f-47d4-bdca-6ecf6d5fa941 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:17.371044 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd84d3f-632f-47d4-bdca-6ecf6d5fa941 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:17.454063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c6a44-f83d-4750-8011-a24ffe6e0183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:17.454763 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c6a44-f83d-4750-8011-a24ffe6e0183 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:17.455118 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c6a44-f83d-4750-8011-a24ffe6e0183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:17.455619 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c6a44-f83d-4750-8011-a24ffe6e0183 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:17.456647 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c6a44-f83d-4750-8011-a24ffe6e0183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:17.458815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c6a44-f83d-4750-8011-a24ffe6e0183 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:17.477926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1175/2332] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:15:17.478425 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96f565-e488-4b74-989d-6869d86ffecc admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:17.506189 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96f565-e488-4b74-989d-6869d86ffecc admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:17.532942 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba2b40-1795-41e6-9f6a-b8104a36b55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:17.533661 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba2b40-1795-41e6-9f6a-b8104a36b55f admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:17.533930 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba2b40-1795-41e6-9f6a-b8104a36b55f admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:17.534246 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba2b40-1795-41e6-9f6a-b8104a36b55f admin admin] PATH_INFO: `/v3/projects/83fe15d16b0c423cb1c440dae6d13ffd/users/190915f69ee04745845ae6dc52892755/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:17.542992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beba2b40-1795-41e6-9f6a-b8104a36b55f admin admin] RBAC: Authorizing `identity:create_grant(project_id=83fe15d16b0c423cb1c440dae6d13ffd, user_id=190915f69ee04745845ae6dc52892755, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:17.545815 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beba2b40-1795-41e6-9f6a-b8104a36b55f admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:17.581170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1158/2333] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:15:17 2022] PUT /identity/v3/projects/83fe15d16b0c423cb1c440dae6d13ffd/users/190915f69ee04745845ae6dc52892755/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:15:17.589116 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b67225-bbcc-4402-9062-139c52974946 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:17.589789 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b67225-bbcc-4402-9062-139c52974946 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:17.590165 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b67225-bbcc-4402-9062-139c52974946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:17.641584 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-a9b67225-bbcc-4402-9062-139c52974946 None None] MFA Rules not processed for user `190915f69ee04745845ae6dc52892755`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:17.643360 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-a9b67225-bbcc-4402-9062-139c52974946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:17.778092 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1176/2334] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:17 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:15:17.786439 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba2b40-1795-41e6-9f6a-b8104a36b55f admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:17.788989 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-beba2b40-1795-41e6-9f6a-b8104a36b55f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:17.880342 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3379fa-3fb5-4dab-afb1-664f26a24672 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:17.880606 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3379fa-3fb5-4dab-afb1-664f26a24672 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:17.881113 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3379fa-3fb5-4dab-afb1-664f26a24672 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:17.882961 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1177/2335] 10.222.0.44 () {54 vars in 921 bytes} [Sun Sep 11 15:15:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 15:15:17.890326 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b38d9c-5097-4e2a-9ca0-0e6f18698a4d None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:17.890636 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b38d9c-5097-4e2a-9ca0-0e6f18698a4d None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:17.890833 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b38d9c-5097-4e2a-9ca0-0e6f18698a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:17.917305 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba2b40-1795-41e6-9f6a-b8104a36b55f admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:17.921875 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-85b38d9c-5097-4e2a-9ca0-0e6f18698a4d None None] MFA Rules not processed for user `453297dac64046d19fc1c331d2d85abc`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:17.923318 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-85b38d9c-5097-4e2a-9ca0-0e6f18698a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:17.923755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0ec5a-5bb9-4e6d-8555-77e7ad5a61d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:17.924459 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0ec5a-5bb9-4e6d-8555-77e7ad5a61d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:17.924810 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0ec5a-5bb9-4e6d-8555-77e7ad5a61d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:17.925108 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0ec5a-5bb9-4e6d-8555-77e7ad5a61d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:17.948730 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c0ec5a-5bb9-4e6d-8555-77e7ad5a61d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:17.951812 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c0ec5a-5bb9-4e6d-8555-77e7ad5a61d7 service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:17.971545 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1159/2336] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:15:17 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:15:18.046362 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1178/2337] 10.222.0.44 () {58 vars in 1018 bytes} [Sun Sep 11 15:15:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:15:18.065279 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0ec5a-5bb9-4e6d-8555-77e7ad5a61d7 service cinder] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:18.068724 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b2c0ec5a-5bb9-4e6d-8555-77e7ad5a61d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:18.115519 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b38d9c-5097-4e2a-9ca0-0e6f18698a4d None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:18.122991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:18.131045 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b38d9c-5097-4e2a-9ca0-0e6f18698a4d None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:18.137288 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005f4bc-23dd-40e7-8c95-0b5c4ea96bfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:18.138396 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005f4bc-23dd-40e7-8c95-0b5c4ea96bfb service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:18.138910 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005f4bc-23dd-40e7-8c95-0b5c4ea96bfb service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:18.139384 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005f4bc-23dd-40e7-8c95-0b5c4ea96bfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:18.202183 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0ec5a-5bb9-4e6d-8555-77e7ad5a61d7 service cinder] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:18.206956 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0542a-b342-4254-9216-cde2d8daa299 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fe5293447346eaafd16818e5bd12f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:18.208276 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0542a-b342-4254-9216-cde2d8daa299 service glance] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:18.209121 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0542a-b342-4254-9216-cde2d8daa299 service glance] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:18.209635 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0542a-b342-4254-9216-cde2d8daa299 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:18.225057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0542a-b342-4254-9216-cde2d8daa299 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:18.226971 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0542a-b342-4254-9216-cde2d8daa299 service glance] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:18.250914 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1160/2338] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:15:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:15:18.298689 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6005f4bc-23dd-40e7-8c95-0b5c4ea96bfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:18.301098 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6005f4bc-23dd-40e7-8c95-0b5c4ea96bfb service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:18.321575 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1179/2339] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:15:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:15:18.416957 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b355fd-53e8-49a7-9d59-2e256eee83b6 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:18.417553 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b355fd-53e8-49a7-9d59-2e256eee83b6 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:18.417952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b355fd-53e8-49a7-9d59-2e256eee83b6 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:18.420462 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1161/2340] 10.222.0.44 () {54 vars in 921 bytes} [Sun Sep 11 15:15:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 15:15:18.429060 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aecf85-942c-47a1-a295-f630aeb9d580 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:18.429577 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aecf85-942c-47a1-a295-f630aeb9d580 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:18.430055 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aecf85-942c-47a1-a295-f630aeb9d580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:18.467839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-f7aecf85-942c-47a1-a295-f630aeb9d580 None None] MFA Rules not processed for user `697b73f182e6419a84e0c8c6b0fc0c06`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:18.467839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-f7aecf85-942c-47a1-a295-f630aeb9d580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:18.584206 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1180/2341] 10.222.0.44 () {58 vars in 1018 bytes} [Sun Sep 11 15:15:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:18.602754 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b355fd-53e8-49a7-9d59-2e256eee83b6 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:18.605539 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-a1b355fd-53e8-49a7-9d59-2e256eee83b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:18.621768 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:18.634582 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b355fd-53e8-49a7-9d59-2e256eee83b6 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:18.640197 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244ae0b-8c8f-4daa-9982-22833ca65cc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:18.640950 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244ae0b-8c8f-4daa-9982-22833ca65cc9 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:18.641374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244ae0b-8c8f-4daa-9982-22833ca65cc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:18.641776 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244ae0b-8c8f-4daa-9982-22833ca65cc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:18.651962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e244ae0b-8c8f-4daa-9982-22833ca65cc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:18.654059 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e244ae0b-8c8f-4daa-9982-22833ca65cc9 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:18.666485 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1162/2342] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:15:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:15:18.835180 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aecf85-942c-47a1-a295-f630aeb9d580 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:18.861642 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee34a4-008a-4226-a17f-7340d521ad75 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:18.862175 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee34a4-008a-4226-a17f-7340d521ad75 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:18.862657 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee34a4-008a-4226-a17f-7340d521ad75 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:18.865425 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1163/2343] 10.222.0.44 () {54 vars in 921 bytes} [Sun Sep 11 15:15:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 15:15:18.874880 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175898f2-7a56-4fb4-873a-8bd22568bd52 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:18.875451 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175898f2-7a56-4fb4-873a-8bd22568bd52 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:18.875753 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175898f2-7a56-4fb4-873a-8bd22568bd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:18.901274 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-175898f2-7a56-4fb4-873a-8bd22568bd52 None None] MFA Rules not processed for user `453297dac64046d19fc1c331d2d85abc`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:18.902694 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-175898f2-7a56-4fb4-873a-8bd22568bd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:18.913754 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1164/2344] 10.222.0.44 () {58 vars in 1018 bytes} [Sun Sep 11 15:15:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:18.928610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175898f2-7a56-4fb4-873a-8bd22568bd52 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:18.931013 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-175898f2-7a56-4fb4-873a-8bd22568bd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:18.950311 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175898f2-7a56-4fb4-873a-8bd22568bd52 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:18.956103 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dad70b-9b71-431b-8144-7822ca19fbfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:18.957166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dad70b-9b71-431b-8144-7822ca19fbfe service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:18.957825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dad70b-9b71-431b-8144-7822ca19fbfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:18.958341 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dad70b-9b71-431b-8144-7822ca19fbfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:18.964040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aecf85-942c-47a1-a295-f630aeb9d580 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:18.969534 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173818b9-dd33-4769-aab2-fb4a5e126fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:18.970212 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173818b9-dd33-4769-aab2-fb4a5e126fda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:18.970496 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173818b9-dd33-4769-aab2-fb4a5e126fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:18.970770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173818b9-dd33-4769-aab2-fb4a5e126fda admin admin] PATH_INFO: `/v3/users/bfa65f323c314aa19332f81d190939d8` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:18.972079 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173818b9-dd33-4769-aab2-fb4a5e126fda admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfa65f323c314aa19332f81d190939d8)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:18.974133 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173818b9-dd33-4769-aab2-fb4a5e126fda admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:18.979189 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dad70b-9b71-431b-8144-7822ca19fbfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:18.981383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dad70b-9b71-431b-8144-7822ca19fbfe service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:19.010817 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1165/2345] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:15:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:15:19.065422 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-173818b9-dd33-4769-aab2-fb4a5e126fda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfa65f323c314aa19332f81d190939d8'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:19.078784 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-173818b9-dd33-4769-aab2-fb4a5e126fda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfa65f323c314aa19332f81d190939d8'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:19.081123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:19.106666 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-173818b9-dd33-4769-aab2-fb4a5e126fda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfa65f323c314aa19332f81d190939d8'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:19.107592 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-173818b9-dd33-4769-aab2-fb4a5e126fda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfa65f323c314aa19332f81d190939d8'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:19.132318 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-173818b9-dd33-4769-aab2-fb4a5e126fda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfa65f323c314aa19332f81d190939d8'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:19.142589 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1181/2346] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:15:18 2022] DELETE /identity/v3/users/bfa65f323c314aa19332f81d190939d8 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:19.149609 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7859cac-de16-4729-96cb-e7049bdb8bcf None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:19.151016 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7859cac-de16-4729-96cb-e7049bdb8bcf None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:19.151285 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7859cac-de16-4729-96cb-e7049bdb8bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:19.179713 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-a7859cac-de16-4729-96cb-e7049bdb8bcf None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:19.180984 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-a7859cac-de16-4729-96cb-e7049bdb8bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:19.285470 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1166/2347] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:15:19.323123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173818b9-dd33-4769-aab2-fb4a5e126fda admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:19.326470 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-173818b9-dd33-4769-aab2-fb4a5e126fda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:19.411160 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173818b9-dd33-4769-aab2-fb4a5e126fda admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:19.415795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b5b9f-1849-4a8a-bcdd-68ecd5672d8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:19.416873 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b5b9f-1849-4a8a-bcdd-68ecd5672d8d service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:19.417377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b5b9f-1849-4a8a-bcdd-68ecd5672d8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:19.417942 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b5b9f-1849-4a8a-bcdd-68ecd5672d8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:19.430113 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b5b9f-1849-4a8a-bcdd-68ecd5672d8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:19.432269 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b5b9f-1849-4a8a-bcdd-68ecd5672d8d service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:19.444335 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1182/2348] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:15:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:15:19.991832 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7859cac-de16-4729-96cb-e7049bdb8bcf None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:19.994287 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-a7859cac-de16-4729-96cb-e7049bdb8bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:20.014990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7859cac-de16-4729-96cb-e7049bdb8bcf None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:20.021085 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292bb90b-c19d-4236-8b53-430ad6c2c734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:20.021911 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bb90b-c19d-4236-8b53-430ad6c2c734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:20.022391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bb90b-c19d-4236-8b53-430ad6c2c734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:20.022809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bb90b-c19d-4236-8b53-430ad6c2c734 admin admin] PATH_INFO: `/v3/projects/fafd2f9bb2f34b20b7280784c9f065a9` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:20.025123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292bb90b-c19d-4236-8b53-430ad6c2c734 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fafd2f9bb2f34b20b7280784c9f065a9)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:20.028958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292bb90b-c19d-4236-8b53-430ad6c2c734 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:20.090204 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-292bb90b-c19d-4236-8b53-430ad6c2c734 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fafd2f9bb2f34b20b7280784c9f065a9', 'request_id': 'req-292bb90b-c19d-4236-8b53-430ad6c2c734'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:20.106529 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-292bb90b-c19d-4236-8b53-430ad6c2c734 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fafd2f9bb2f34b20b7280784c9f065a9', 'request_id': 'req-292bb90b-c19d-4236-8b53-430ad6c2c734'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:20.119169 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-292bb90b-c19d-4236-8b53-430ad6c2c734 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fafd2f9bb2f34b20b7280784c9f065a9', 'request_id': 'req-292bb90b-c19d-4236-8b53-430ad6c2c734'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:20.138176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-292bb90b-c19d-4236-8b53-430ad6c2c734 admin admin] The token cache is being invalidate because project fafd2f9bb2f34b20b7280784c9f065a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:15:20.139219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-292bb90b-c19d-4236-8b53-430ad6c2c734 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:20.140765 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-292bb90b-c19d-4236-8b53-430ad6c2c734 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:20.143798 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1167/2349] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:15:19 2022] DELETE /identity/v3/projects/fafd2f9bb2f34b20b7280784c9f065a9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:20.148763 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b5b9f-1849-4a8a-bcdd-68ecd5672d8d service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:20.167264 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b5b9f-1849-4a8a-bcdd-68ecd5672d8d service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:20.257252 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf3ed0-79a9-4dc3-bbc3-de9d99c0efe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:20.259327 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf3ed0-79a9-4dc3-bbc3-de9d99c0efe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:20.260239 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf3ed0-79a9-4dc3-bbc3-de9d99c0efe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:20.261059 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf3ed0-79a9-4dc3-bbc3-de9d99c0efe3 admin admin] PATH_INFO: `/v3/users/190915f69ee04745845ae6dc52892755` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:20.263765 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf3ed0-79a9-4dc3-bbc3-de9d99c0efe3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=190915f69ee04745845ae6dc52892755)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:20.267942 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf3ed0-79a9-4dc3-bbc3-de9d99c0efe3 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:20.373195 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-ecbf3ed0-79a9-4dc3-bbc3-de9d99c0efe3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '190915f69ee04745845ae6dc52892755'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:20.387794 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-ecbf3ed0-79a9-4dc3-bbc3-de9d99c0efe3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '190915f69ee04745845ae6dc52892755'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:20.410114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-ecbf3ed0-79a9-4dc3-bbc3-de9d99c0efe3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '190915f69ee04745845ae6dc52892755'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:20.410665 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-ecbf3ed0-79a9-4dc3-bbc3-de9d99c0efe3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '190915f69ee04745845ae6dc52892755'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:20.416555 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:20.420752 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-ecbf3ed0-79a9-4dc3-bbc3-de9d99c0efe3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '190915f69ee04745845ae6dc52892755'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:20.431238 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1183/2350] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:15:20 2022] DELETE /identity/v3/users/190915f69ee04745845ae6dc52892755 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:15:21.133451 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292bb90b-c19d-4236-8b53-430ad6c2c734 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:21.135588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-292bb90b-c19d-4236-8b53-430ad6c2c734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:21.162462 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292bb90b-c19d-4236-8b53-430ad6c2c734 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:21.281913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:21.289027 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003fd5c-8983-465a-80d0-ca4fdba77152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:21.289817 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003fd5c-8983-465a-80d0-ca4fdba77152 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:21.291071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003fd5c-8983-465a-80d0-ca4fdba77152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:21.291513 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003fd5c-8983-465a-80d0-ca4fdba77152 admin admin] PATH_INFO: `/v3/projects/83fe15d16b0c423cb1c440dae6d13ffd` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:21.292997 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003fd5c-8983-465a-80d0-ca4fdba77152 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83fe15d16b0c423cb1c440dae6d13ffd)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:21.295245 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003fd5c-8983-465a-80d0-ca4fdba77152 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:21.382701 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-f003fd5c-8983-465a-80d0-ca4fdba77152 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83fe15d16b0c423cb1c440dae6d13ffd', 'request_id': 'req-f003fd5c-8983-465a-80d0-ca4fdba77152'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:21.395611 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-f003fd5c-8983-465a-80d0-ca4fdba77152 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83fe15d16b0c423cb1c440dae6d13ffd', 'request_id': 'req-f003fd5c-8983-465a-80d0-ca4fdba77152'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:21.406065 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-f003fd5c-8983-465a-80d0-ca4fdba77152 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83fe15d16b0c423cb1c440dae6d13ffd', 'request_id': 'req-f003fd5c-8983-465a-80d0-ca4fdba77152'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:21.421657 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-f003fd5c-8983-465a-80d0-ca4fdba77152 admin admin] The token cache is being invalidate because project 83fe15d16b0c423cb1c440dae6d13ffd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:15:21.421950 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-f003fd5c-8983-465a-80d0-ca4fdba77152 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:21.422618 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-f003fd5c-8983-465a-80d0-ca4fdba77152 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:21.424781 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1168/2351] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:15:21 2022] DELETE /identity/v3/projects/83fe15d16b0c423cb1c440dae6d13ffd => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:21.429930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4705fc8-3504-4d22-9eb4-8206ae17433a None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:21.430523 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4705fc8-3504-4d22-9eb4-8206ae17433a None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:21.431236 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4705fc8-3504-4d22-9eb4-8206ae17433a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:21.454985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-c4705fc8-3504-4d22-9eb4-8206ae17433a None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:21.456349 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-c4705fc8-3504-4d22-9eb4-8206ae17433a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:21.541517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1184/2352] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:21.545069 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e652d6f-3aac-4bc5-9dff-7aa22ad9088e None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:21.546652 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e652d6f-3aac-4bc5-9dff-7aa22ad9088e None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:21.547267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e652d6f-3aac-4bc5-9dff-7aa22ad9088e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:21.581501 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-9e652d6f-3aac-4bc5-9dff-7aa22ad9088e None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:21.583064 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-9e652d6f-3aac-4bc5-9dff-7aa22ad9088e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:21.598201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1169/2353] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:15:21.602451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4705fc8-3504-4d22-9eb4-8206ae17433a None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:21.619214 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4705fc8-3504-4d22-9eb4-8206ae17433a None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:21.628834 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce3642-f712-440c-872c-6e7b601fe201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:21.629919 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce3642-f712-440c-872c-6e7b601fe201 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:21.630394 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce3642-f712-440c-872c-6e7b601fe201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:21.630913 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce3642-f712-440c-872c-6e7b601fe201 admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:21.632209 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce3642-f712-440c-872c-6e7b601fe201 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:15:21.632686 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce3642-f712-440c-872c-6e7b601fe201 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:21.636027 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce3642-f712-440c-872c-6e7b601fe201 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:21.657500 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1185/2354] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:15:21.664709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e652d6f-3aac-4bc5-9dff-7aa22ad9088e None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:21.678612 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e652d6f-3aac-4bc5-9dff-7aa22ad9088e None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:21.686766 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1bf71-9968-4155-a908-468504e39279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:21.687766 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1bf71-9968-4155-a908-468504e39279 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:21.688251 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1bf71-9968-4155-a908-468504e39279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:21.688741 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1bf71-9968-4155-a908-468504e39279 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:21.689834 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1bf71-9968-4155-a908-468504e39279 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:21.693340 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1bf71-9968-4155-a908-468504e39279 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:21.739185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1170/2355] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:15:21 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:15:21.743496 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce3642-f712-440c-872c-6e7b601fe201 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:21.755640 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce3642-f712-440c-872c-6e7b601fe201 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:21.829885 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27275b95-6d29-4d84-8f73-af7a4d8f8c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:21.830605 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27275b95-6d29-4d84-8f73-af7a4d8f8c70 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:21.831063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27275b95-6d29-4d84-8f73-af7a4d8f8c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:21.831393 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27275b95-6d29-4d84-8f73-af7a4d8f8c70 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:21.832109 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27275b95-6d29-4d84-8f73-af7a4d8f8c70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:21.834207 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27275b95-6d29-4d84-8f73-af7a4d8f8c70 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:21.867277 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1186/2356] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:15:21 2022] POST /identity/v3/users => generated 287 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:15:21.872406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1bf71-9968-4155-a908-468504e39279 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:21.890569 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1bf71-9968-4155-a908-468504e39279 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:21.900967 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34597050-8f23-405f-98bc-828cbff0db00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:21.902134 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34597050-8f23-405f-98bc-828cbff0db00 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:21.902887 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34597050-8f23-405f-98bc-828cbff0db00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:21.903293 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34597050-8f23-405f-98bc-828cbff0db00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:21.904386 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34597050-8f23-405f-98bc-828cbff0db00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:21.907469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34597050-8f23-405f-98bc-828cbff0db00 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:21.931134 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1171/2357] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:15:21.935494 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27275b95-6d29-4d84-8f73-af7a4d8f8c70 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:21.953011 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27275b95-6d29-4d84-8f73-af7a4d8f8c70 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:21.960681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50347e8c-8b42-46e9-b792-002680e64da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:21.961766 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50347e8c-8b42-46e9-b792-002680e64da2 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:21.962170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50347e8c-8b42-46e9-b792-002680e64da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:21.962589 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50347e8c-8b42-46e9-b792-002680e64da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:21.963865 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50347e8c-8b42-46e9-b792-002680e64da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:21.967092 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50347e8c-8b42-46e9-b792-002680e64da2 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:21.982899 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1187/2358] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:15:21.988508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34597050-8f23-405f-98bc-828cbff0db00 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:22.001539 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34597050-8f23-405f-98bc-828cbff0db00 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:22.007410 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14319ca2-e461-422d-aee3-18681d7f9e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:22.008389 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14319ca2-e461-422d-aee3-18681d7f9e53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:22.009521 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14319ca2-e461-422d-aee3-18681d7f9e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:22.009994 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14319ca2-e461-422d-aee3-18681d7f9e53 admin admin] PATH_INFO: `/v3/projects/285a6a1161744beaba015a92f1457fe2/users/bcff00d9946442599a2f131d7c3e43d8/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:22.055512 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14319ca2-e461-422d-aee3-18681d7f9e53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=285a6a1161744beaba015a92f1457fe2, user_id=bcff00d9946442599a2f131d7c3e43d8, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:22.060118 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14319ca2-e461-422d-aee3-18681d7f9e53 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:22.088565 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1172/2359] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:15:21 2022] PUT /identity/v3/projects/285a6a1161744beaba015a92f1457fe2/users/bcff00d9946442599a2f131d7c3e43d8/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:22.090511 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8dd64-fc12-42e5-8c23-c86ff9df978f None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:22.091023 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8dd64-fc12-42e5-8c23-c86ff9df978f None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:22.091472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8dd64-fc12-42e5-8c23-c86ff9df978f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:22.125446 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-a1e8dd64-fc12-42e5-8c23-c86ff9df978f None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:22.126396 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-a1e8dd64-fc12-42e5-8c23-c86ff9df978f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:22.199743 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:22.219758 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1188/2360] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:22.231166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14319ca2-e461-422d-aee3-18681d7f9e53 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:22.235268 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-14319ca2-e461-422d-aee3-18681d7f9e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:22.359589 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14319ca2-e461-422d-aee3-18681d7f9e53 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:22.366900 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bde95c-7fc3-4c85-beb0-4ed8134f111a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:22.368007 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bde95c-7fc3-4c85-beb0-4ed8134f111a service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:22.368475 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bde95c-7fc3-4c85-beb0-4ed8134f111a service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:22.368970 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bde95c-7fc3-4c85-beb0-4ed8134f111a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:22.386717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bde95c-7fc3-4c85-beb0-4ed8134f111a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:22.389557 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bde95c-7fc3-4c85-beb0-4ed8134f111a service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:22.409512 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1173/2361] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:15:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:15:23.526098 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65b165-55d1-4613-aac2-5cb7daa25e39 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:23.526915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65b165-55d1-4613-aac2-5cb7daa25e39 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:23.526915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65b165-55d1-4613-aac2-5cb7daa25e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:23.587195 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-cb65b165-55d1-4613-aac2-5cb7daa25e39 None None] MFA Rules not processed for user `bcff00d9946442599a2f131d7c3e43d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:23.588617 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-cb65b165-55d1-4613-aac2-5cb7daa25e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:23.670728 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1189/2362] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:15:23.676787 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bde95c-7fc3-4c85-beb0-4ed8134f111a service neutron] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:23.694383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bde95c-7fc3-4c85-beb0-4ed8134f111a service neutron] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:23.702626 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379664d5-357c-4029-9ba1-e418766b1337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:23.703386 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379664d5-357c-4029-9ba1-e418766b1337 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:23.703854 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379664d5-357c-4029-9ba1-e418766b1337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:23.704316 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379664d5-357c-4029-9ba1-e418766b1337 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:23.705451 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379664d5-357c-4029-9ba1-e418766b1337 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:23.709386 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379664d5-357c-4029-9ba1-e418766b1337 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:23.735398 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:23.750164 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1174/2363] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:15:23 2022] POST /identity/v3/projects => generated 373 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:15:23.753972 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb65b165-55d1-4613-aac2-5cb7daa25e39 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:23.767665 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb65b165-55d1-4613-aac2-5cb7daa25e39 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:23.836091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f707b-6bc4-4c7d-8add-34f4e7faca1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:23.836935 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f707b-6bc4-4c7d-8add-34f4e7faca1a admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:23.837358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f707b-6bc4-4c7d-8add-34f4e7faca1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:23.837750 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f707b-6bc4-4c7d-8add-34f4e7faca1a admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:23.838631 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f707b-6bc4-4c7d-8add-34f4e7faca1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:23.840788 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f707b-6bc4-4c7d-8add-34f4e7faca1a admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:23.867005 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1190/2364] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:15:23 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:15:23.873798 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379664d5-357c-4029-9ba1-e418766b1337 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:23.891987 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379664d5-357c-4029-9ba1-e418766b1337 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:23.900209 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969c028b-b516-4e3b-b0ee-1e1e84c0c428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:23.901320 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969c028b-b516-4e3b-b0ee-1e1e84c0c428 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:23.901778 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969c028b-b516-4e3b-b0ee-1e1e84c0c428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:23.902230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969c028b-b516-4e3b-b0ee-1e1e84c0c428 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:23.903418 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969c028b-b516-4e3b-b0ee-1e1e84c0c428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:23.906490 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969c028b-b516-4e3b-b0ee-1e1e84c0c428 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:23.925002 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1175/2365] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:15:23.929684 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f707b-6bc4-4c7d-8add-34f4e7faca1a admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:23.942762 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f707b-6bc4-4c7d-8add-34f4e7faca1a admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:23.948316 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d139856-08fe-4128-958f-e1a0a45f2b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:23.949288 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d139856-08fe-4128-958f-e1a0a45f2b69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:23.949600 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d139856-08fe-4128-958f-e1a0a45f2b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:23.949972 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d139856-08fe-4128-958f-e1a0a45f2b69 admin admin] PATH_INFO: `/v3/projects/60733350088a4a7e99ebac8a6fc053c4/users/e5f12516e4164741979e2d3cf0b9deae/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:23.968564 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d139856-08fe-4128-958f-e1a0a45f2b69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60733350088a4a7e99ebac8a6fc053c4, user_id=e5f12516e4164741979e2d3cf0b9deae, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:23.971469 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d139856-08fe-4128-958f-e1a0a45f2b69 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:23.986914 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1191/2366] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:15:23 2022] PUT /identity/v3/projects/60733350088a4a7e99ebac8a6fc053c4/users/e5f12516e4164741979e2d3cf0b9deae/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:23.991898 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969c028b-b516-4e3b-b0ee-1e1e84c0c428 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:24.003624 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969c028b-b516-4e3b-b0ee-1e1e84c0c428 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:24.074449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce944f2-37db-4ff0-978c-3a0a4cccf7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:24.075119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce944f2-37db-4ff0-978c-3a0a4cccf7ea admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:24.075530 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce944f2-37db-4ff0-978c-3a0a4cccf7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:24.075766 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce944f2-37db-4ff0-978c-3a0a4cccf7ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:24.076453 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce944f2-37db-4ff0-978c-3a0a4cccf7ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:24.078355 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce944f2-37db-4ff0-978c-3a0a4cccf7ea admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:24.090838 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1176/2367] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:15:24.095666 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d139856-08fe-4128-958f-e1a0a45f2b69 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:24.106981 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d139856-08fe-4128-958f-e1a0a45f2b69 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:24.112223 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17443d8-ff61-4741-8f0c-e16b3b175ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:24.112907 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17443d8-ff61-4741-8f0c-e16b3b175ce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:24.113214 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17443d8-ff61-4741-8f0c-e16b3b175ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:24.113488 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17443d8-ff61-4741-8f0c-e16b3b175ce7 admin admin] PATH_INFO: `/v3/projects/60733350088a4a7e99ebac8a6fc053c4/users/e5f12516e4164741979e2d3cf0b9deae/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:24.115444 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17443d8-ff61-4741-8f0c-e16b3b175ce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60733350088a4a7e99ebac8a6fc053c4, user_id=e5f12516e4164741979e2d3cf0b9deae, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:24.118142 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17443d8-ff61-4741-8f0c-e16b3b175ce7 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:24.130739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1192/2368] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:15:24 2022] PUT /identity/v3/projects/60733350088a4a7e99ebac8a6fc053c4/users/e5f12516e4164741979e2d3cf0b9deae/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:15:24.228206 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061311b-b4b6-47f0-b4e2-aee89c54b5e6 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:24.228478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061311b-b4b6-47f0-b4e2-aee89c54b5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:24.228746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061311b-b4b6-47f0-b4e2-aee89c54b5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:24.257960 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-a061311b-b4b6-47f0-b4e2-aee89c54b5e6 None None] MFA Rules not processed for user `3fada78ceaeb4f3bad77c516f6dfcfd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:24.259521 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-a061311b-b4b6-47f0-b4e2-aee89c54b5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:24.343916 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1177/2369] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 15:15:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:24.363256 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17443d8-ff61-4741-8f0c-e16b3b175ce7 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:24.366764 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-d17443d8-ff61-4741-8f0c-e16b3b175ce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:24.449947 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:24.485815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17443d8-ff61-4741-8f0c-e16b3b175ce7 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:24.493693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedea29c-7ad2-44e3-a40a-1031b91976d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05aa5e40ee4542719657bc03b2015e72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:24.494485 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedea29c-7ad2-44e3-a40a-1031b91976d8 service swift] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:24.495470 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedea29c-7ad2-44e3-a40a-1031b91976d8 service swift] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:24.495739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedea29c-7ad2-44e3-a40a-1031b91976d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:24.509561 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedea29c-7ad2-44e3-a40a-1031b91976d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:24.511395 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedea29c-7ad2-44e3-a40a-1031b91976d8 service swift] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:24.523772 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1193/2370] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 15:15:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 15:15:25.534050 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193599e0-6f94-4c4f-a673-72743822653f None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:25.534929 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193599e0-6f94-4c4f-a673-72743822653f None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:25.534929 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193599e0-6f94-4c4f-a673-72743822653f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:25.576236 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-193599e0-6f94-4c4f-a673-72743822653f None None] MFA Rules not processed for user `e5f12516e4164741979e2d3cf0b9deae`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:25.578283 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-193599e0-6f94-4c4f-a673-72743822653f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:25.689257 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1178/2371] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:25.704618 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedea29c-7ad2-44e3-a40a-1031b91976d8 service swift] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:25.706168 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-cedea29c-7ad2-44e3-a40a-1031b91976d8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:25.824756 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedea29c-7ad2-44e3-a40a-1031b91976d8 service swift] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:25.829349 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e945e41-e7d9-4ff2-b4d5-d348e9592927 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:25.830089 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e945e41-e7d9-4ff2-b4d5-d348e9592927 service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:25.830368 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e945e41-e7d9-4ff2-b4d5-d348e9592927 service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:25.830844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e945e41-e7d9-4ff2-b4d5-d348e9592927 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:25.954701 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e945e41-e7d9-4ff2-b4d5-d348e9592927 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:25.956785 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e945e41-e7d9-4ff2-b4d5-d348e9592927 service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:25.984882 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1194/2372] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:15:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:15:27.755809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193599e0-6f94-4c4f-a673-72743822653f None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:27.757578 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-193599e0-6f94-4c4f-a673-72743822653f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:27.784221 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193599e0-6f94-4c4f-a673-72743822653f None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:27.789632 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a81cdf-bf8a-44fb-90d3-379f221fb41e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:27.790788 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a81cdf-bf8a-44fb-90d3-379f221fb41e service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:27.791374 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a81cdf-bf8a-44fb-90d3-379f221fb41e service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:27.791876 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a81cdf-bf8a-44fb-90d3-379f221fb41e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:27.825949 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a81cdf-bf8a-44fb-90d3-379f221fb41e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:27.837337 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a81cdf-bf8a-44fb-90d3-379f221fb41e service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:27.859165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1179/2373] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:15:27 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:15:35.969548 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=61 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:37.849572 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:37.959921 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e945e41-e7d9-4ff2-b4d5-d348e9592927 service cinder] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:37.962370 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-9e945e41-e7d9-4ff2-b4d5-d348e9592927 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:38.063278 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e945e41-e7d9-4ff2-b4d5-d348e9592927 service cinder] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:38.070474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba38c48-6ca3-4207-8bf8-af3c8407148f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead06362f8a2444bbfe1405cd602f1c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:38.071471 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba38c48-6ca3-4207-8bf8-af3c8407148f service placement] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:38.071884 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba38c48-6ca3-4207-8bf8-af3c8407148f service placement] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:38.072246 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba38c48-6ca3-4207-8bf8-af3c8407148f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:38.074470 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-4ba38c48-6ca3-4207-8bf8-af3c8407148f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:38.087810 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba38c48-6ca3-4207-8bf8-af3c8407148f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:38.089485 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba38c48-6ca3-4207-8bf8-af3c8407148f service placement] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:38.100652 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1195/2374] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:15:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:15:42.673883 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a81cdf-bf8a-44fb-90d3-379f221fb41e service cinder] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:42.747832 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a81cdf-bf8a-44fb-90d3-379f221fb41e service cinder] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:42.753782 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1af20-deaa-4ca2-9977-0e9c7cab4c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:42.754443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1af20-deaa-4ca2-9977-0e9c7cab4c96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:42.754725 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1af20-deaa-4ca2-9977-0e9c7cab4c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:42.755016 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1af20-deaa-4ca2-9977-0e9c7cab4c96 admin admin] PATH_INFO: `/v3/users/bcff00d9946442599a2f131d7c3e43d8` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:42.756202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1af20-deaa-4ca2-9977-0e9c7cab4c96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcff00d9946442599a2f131d7c3e43d8)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:42.758458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1af20-deaa-4ca2-9977-0e9c7cab4c96 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:42.813482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-e9f1af20-deaa-4ca2-9977-0e9c7cab4c96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcff00d9946442599a2f131d7c3e43d8'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:42.828026 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-e9f1af20-deaa-4ca2-9977-0e9c7cab4c96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcff00d9946442599a2f131d7c3e43d8'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:42.842632 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-e9f1af20-deaa-4ca2-9977-0e9c7cab4c96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcff00d9946442599a2f131d7c3e43d8'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:42.843451 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-e9f1af20-deaa-4ca2-9977-0e9c7cab4c96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcff00d9946442599a2f131d7c3e43d8'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:42.852525 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-e9f1af20-deaa-4ca2-9977-0e9c7cab4c96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcff00d9946442599a2f131d7c3e43d8'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:42.859657 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1180/2375] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:15:42 2022] DELETE /identity/v3/users/bcff00d9946442599a2f131d7c3e43d8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:15:43.103145 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba38c48-6ca3-4207-8bf8-af3c8407148f service placement] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:43.121795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba38c48-6ca3-4207-8bf8-af3c8407148f service placement] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:43.234274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9369e0-e2b7-4010-a679-a8f3c59cdcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:43.235399 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9369e0-e2b7-4010-a679-a8f3c59cdcdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:43.235893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9369e0-e2b7-4010-a679-a8f3c59cdcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:43.236437 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9369e0-e2b7-4010-a679-a8f3c59cdcdb admin admin] PATH_INFO: `/v3/projects/285a6a1161744beaba015a92f1457fe2` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:43.238751 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9369e0-e2b7-4010-a679-a8f3c59cdcdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=285a6a1161744beaba015a92f1457fe2)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:43.242535 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9369e0-e2b7-4010-a679-a8f3c59cdcdb admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:43.328016 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-dd9369e0-e2b7-4010-a679-a8f3c59cdcdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '285a6a1161744beaba015a92f1457fe2', 'request_id': 'req-dd9369e0-e2b7-4010-a679-a8f3c59cdcdb'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:43.340348 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-dd9369e0-e2b7-4010-a679-a8f3c59cdcdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '285a6a1161744beaba015a92f1457fe2', 'request_id': 'req-dd9369e0-e2b7-4010-a679-a8f3c59cdcdb'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:43.350809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-dd9369e0-e2b7-4010-a679-a8f3c59cdcdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '285a6a1161744beaba015a92f1457fe2', 'request_id': 'req-dd9369e0-e2b7-4010-a679-a8f3c59cdcdb'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:43.369494 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-dd9369e0-e2b7-4010-a679-a8f3c59cdcdb admin admin] The token cache is being invalidate because project 285a6a1161744beaba015a92f1457fe2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:15:43.369826 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-dd9369e0-e2b7-4010-a679-a8f3c59cdcdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:43.377773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-dd9369e0-e2b7-4010-a679-a8f3c59cdcdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:43.377773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1196/2376] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:15:43 2022] DELETE /identity/v3/projects/285a6a1161744beaba015a92f1457fe2 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:43.387206 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1af20-deaa-4ca2-9977-0e9c7cab4c96 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:43.392003 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-e9f1af20-deaa-4ca2-9977-0e9c7cab4c96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:43.411022 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1af20-deaa-4ca2-9977-0e9c7cab4c96 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:43.504741 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24eb7f4-528f-4776-8123-1e2d31b588fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:43.505641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24eb7f4-528f-4776-8123-1e2d31b588fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:43.505907 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24eb7f4-528f-4776-8123-1e2d31b588fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:43.506199 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24eb7f4-528f-4776-8123-1e2d31b588fb admin admin] PATH_INFO: `/v3/users/e5f12516e4164741979e2d3cf0b9deae` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:43.507454 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24eb7f4-528f-4776-8123-1e2d31b588fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5f12516e4164741979e2d3cf0b9deae)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:43.509767 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24eb7f4-528f-4776-8123-1e2d31b588fb admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:43.537556 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:43.585402 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c24eb7f4-528f-4776-8123-1e2d31b588fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5f12516e4164741979e2d3cf0b9deae'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:43.597746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c24eb7f4-528f-4776-8123-1e2d31b588fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5f12516e4164741979e2d3cf0b9deae'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:43.618649 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c24eb7f4-528f-4776-8123-1e2d31b588fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5f12516e4164741979e2d3cf0b9deae'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:43.619798 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c24eb7f4-528f-4776-8123-1e2d31b588fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5f12516e4164741979e2d3cf0b9deae'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:43.633377 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c24eb7f4-528f-4776-8123-1e2d31b588fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5f12516e4164741979e2d3cf0b9deae'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:43.642426 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1181/2377] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:15:43 2022] DELETE /identity/v3/users/e5f12516e4164741979e2d3cf0b9deae => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:44.258477 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9369e0-e2b7-4010-a679-a8f3c59cdcdb admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:44.260850 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-dd9369e0-e2b7-4010-a679-a8f3c59cdcdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:44.284974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9369e0-e2b7-4010-a679-a8f3c59cdcdb admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:44.311845 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:44.387700 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfb86b-b297-4267-9079-63ff8c351968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:44.390054 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfb86b-b297-4267-9079-63ff8c351968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:44.390891 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfb86b-b297-4267-9079-63ff8c351968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:44.391783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfb86b-b297-4267-9079-63ff8c351968 admin admin] PATH_INFO: `/v3/projects/60733350088a4a7e99ebac8a6fc053c4` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:44.393100 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcfb86b-b297-4267-9079-63ff8c351968 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60733350088a4a7e99ebac8a6fc053c4)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:44.395431 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcfb86b-b297-4267-9079-63ff8c351968 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:44.476885 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-7bcfb86b-b297-4267-9079-63ff8c351968 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60733350088a4a7e99ebac8a6fc053c4', 'request_id': 'req-7bcfb86b-b297-4267-9079-63ff8c351968'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:44.483846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-7bcfb86b-b297-4267-9079-63ff8c351968 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60733350088a4a7e99ebac8a6fc053c4', 'request_id': 'req-7bcfb86b-b297-4267-9079-63ff8c351968'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:44.492171 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-7bcfb86b-b297-4267-9079-63ff8c351968 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60733350088a4a7e99ebac8a6fc053c4', 'request_id': 'req-7bcfb86b-b297-4267-9079-63ff8c351968'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:44.523098 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-7bcfb86b-b297-4267-9079-63ff8c351968 admin admin] The token cache is being invalidate because project 60733350088a4a7e99ebac8a6fc053c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:15:44.523542 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-7bcfb86b-b297-4267-9079-63ff8c351968 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:44.524372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-7bcfb86b-b297-4267-9079-63ff8c351968 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:44.526244 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1197/2378] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:15:44 2022] DELETE /identity/v3/projects/60733350088a4a7e99ebac8a6fc053c4 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:44.531996 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4511fa-f95d-4c05-a448-3546531c3056 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:44.532887 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4511fa-f95d-4c05-a448-3546531c3056 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:44.532887 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4511fa-f95d-4c05-a448-3546531c3056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:44.559633 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-fd4511fa-f95d-4c05-a448-3546531c3056 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:44.561408 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-fd4511fa-f95d-4c05-a448-3546531c3056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:44.631655 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1182/2379] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:44.641863 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca824cf0-e794-450e-9081-dd4fe36b1b3e None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:44.642371 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca824cf0-e794-450e-9081-dd4fe36b1b3e None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:44.642911 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca824cf0-e794-450e-9081-dd4fe36b1b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:44.674688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-ca824cf0-e794-450e-9081-dd4fe36b1b3e None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:44.675934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-ca824cf0-e794-450e-9081-dd4fe36b1b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:44.685792 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1198/2380] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:44.691477 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4511fa-f95d-4c05-a448-3546531c3056 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:44.703450 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4511fa-f95d-4c05-a448-3546531c3056 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:44.713076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c1c96-d90a-4be6-9627-e7907f0aebd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:44.714375 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c1c96-d90a-4be6-9627-e7907f0aebd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:44.714961 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c1c96-d90a-4be6-9627-e7907f0aebd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:44.715330 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c1c96-d90a-4be6-9627-e7907f0aebd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:44.716164 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c1c96-d90a-4be6-9627-e7907f0aebd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:15:44.716549 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c1c96-d90a-4be6-9627-e7907f0aebd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:44.718375 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c1c96-d90a-4be6-9627-e7907f0aebd2 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:44.732976 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1183/2381] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:15:44.740163 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca824cf0-e794-450e-9081-dd4fe36b1b3e None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:44.757991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca824cf0-e794-450e-9081-dd4fe36b1b3e None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:44.766384 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa41860-eb5b-4970-ac2d-1286a8c9c60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:44.767482 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa41860-eb5b-4970-ac2d-1286a8c9c60f admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:44.768015 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa41860-eb5b-4970-ac2d-1286a8c9c60f admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:44.768464 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa41860-eb5b-4970-ac2d-1286a8c9c60f admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:44.769657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa41860-eb5b-4970-ac2d-1286a8c9c60f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:44.773416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa41860-eb5b-4970-ac2d-1286a8c9c60f admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:44.822857 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1199/2382] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:15:44 2022] POST /identity/v3/projects => generated 371 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:15:44.824790 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c1c96-d90a-4be6-9627-e7907f0aebd2 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:44.842608 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c1c96-d90a-4be6-9627-e7907f0aebd2 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:44.919200 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074cee6-7ac4-45c5-9298-ab0092b30223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:44.919832 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074cee6-7ac4-45c5-9298-ab0092b30223 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:44.920108 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074cee6-7ac4-45c5-9298-ab0092b30223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:44.920378 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074cee6-7ac4-45c5-9298-ab0092b30223 admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:44.921186 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6074cee6-7ac4-45c5-9298-ab0092b30223 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:44.923226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6074cee6-7ac4-45c5-9298-ab0092b30223 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:44.949903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1184/2383] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:15:44 2022] POST /identity/v3/users => generated 286 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:15:44.954478 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa41860-eb5b-4970-ac2d-1286a8c9c60f admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:44.967516 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa41860-eb5b-4970-ac2d-1286a8c9c60f admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:44.974143 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d6c7c6-3a04-412f-9616-db298cc71bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:44.975217 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6c7c6-3a04-412f-9616-db298cc71bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:44.975593 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6c7c6-3a04-412f-9616-db298cc71bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:44.975923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6c7c6-3a04-412f-9616-db298cc71bd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:44.976630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d6c7c6-3a04-412f-9616-db298cc71bd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:44.978721 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d6c7c6-3a04-412f-9616-db298cc71bd4 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:44.991763 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1200/2384] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:15:44.996528 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074cee6-7ac4-45c5-9298-ab0092b30223 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:45.007801 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074cee6-7ac4-45c5-9298-ab0092b30223 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:45.013541 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1cbe0-67c6-4edb-84e6-4f7e5f566f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:45.014195 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1cbe0-67c6-4edb-84e6-4f7e5f566f13 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:45.014487 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1cbe0-67c6-4edb-84e6-4f7e5f566f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:45.014765 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1cbe0-67c6-4edb-84e6-4f7e5f566f13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:45.015480 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f1cbe0-67c6-4edb-84e6-4f7e5f566f13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:45.017282 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f1cbe0-67c6-4edb-84e6-4f7e5f566f13 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:45.030504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1185/2385] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:15:45.036823 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d6c7c6-3a04-412f-9616-db298cc71bd4 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:45.049417 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d6c7c6-3a04-412f-9616-db298cc71bd4 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:45.057104 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3f079-3c15-4e1f-9b6c-74152e27cc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:45.058252 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3f079-3c15-4e1f-9b6c-74152e27cc57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:45.058728 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3f079-3c15-4e1f-9b6c-74152e27cc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:45.059190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3f079-3c15-4e1f-9b6c-74152e27cc57 admin admin] PATH_INFO: `/v3/projects/3300ef591ba14ba0a1d1c437ce613702/users/70e3f3481d784b338f618d6cc1541600/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:45.087817 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee3f079-3c15-4e1f-9b6c-74152e27cc57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3300ef591ba14ba0a1d1c437ce613702, user_id=70e3f3481d784b338f618d6cc1541600, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:45.093358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee3f079-3c15-4e1f-9b6c-74152e27cc57 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:45.116774 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1201/2386] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:15:45 2022] PUT /identity/v3/projects/3300ef591ba14ba0a1d1c437ce613702/users/70e3f3481d784b338f618d6cc1541600/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:45.136616 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effed068-0514-424c-bb5a-b4a14b2e6d2d None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:45.137014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effed068-0514-424c-bb5a-b4a14b2e6d2d None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:45.137299 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effed068-0514-424c-bb5a-b4a14b2e6d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:45.147877 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:45.178436 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-effed068-0514-424c-bb5a-b4a14b2e6d2d None None] MFA Rules not processed for user `70e3f3481d784b338f618d6cc1541600`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:45.179687 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-effed068-0514-424c-bb5a-b4a14b2e6d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:45.258722 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1186/2387] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:45.273380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3f079-3c15-4e1f-9b6c-74152e27cc57 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:45.276782 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-aee3f079-3c15-4e1f-9b6c-74152e27cc57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:45.343348 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:45.351497 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3f079-3c15-4e1f-9b6c-74152e27cc57 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:45.358372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ef57e-695d-4304-9261-5faa630da40d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:45.359448 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ef57e-695d-4304-9261-5faa630da40d service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:45.360301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ef57e-695d-4304-9261-5faa630da40d service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:45.361016 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ef57e-695d-4304-9261-5faa630da40d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:45.372515 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ef57e-695d-4304-9261-5faa630da40d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:45.374035 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ef57e-695d-4304-9261-5faa630da40d service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:45.384900 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1202/2388] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:15:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:15:45.408504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effed068-0514-424c-bb5a-b4a14b2e6d2d None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:45.498336 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effed068-0514-424c-bb5a-b4a14b2e6d2d None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:45.506170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fbc768-1f01-4986-a373-266e01f30015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:45.507251 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbc768-1f01-4986-a373-266e01f30015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:45.507734 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbc768-1f01-4986-a373-266e01f30015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:45.508220 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbc768-1f01-4986-a373-266e01f30015 admin admin] PATH_INFO: `/v3/users/70e3f3481d784b338f618d6cc1541600` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:45.510526 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fbc768-1f01-4986-a373-266e01f30015 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70e3f3481d784b338f618d6cc1541600)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:45.514507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fbc768-1f01-4986-a373-266e01f30015 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:45.583918 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-e7fbc768-1f01-4986-a373-266e01f30015 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70e3f3481d784b338f618d6cc1541600'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:45.593235 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-e7fbc768-1f01-4986-a373-266e01f30015 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70e3f3481d784b338f618d6cc1541600'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:45.606876 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-e7fbc768-1f01-4986-a373-266e01f30015 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70e3f3481d784b338f618d6cc1541600'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:45.607610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-e7fbc768-1f01-4986-a373-266e01f30015 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70e3f3481d784b338f618d6cc1541600'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:45.615288 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-e7fbc768-1f01-4986-a373-266e01f30015 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70e3f3481d784b338f618d6cc1541600'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:45.624953 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1187/2389] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:15:45 2022] DELETE /identity/v3/users/70e3f3481d784b338f618d6cc1541600 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:45.629949 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02af28-37a6-4eac-b83f-f6210651fe87 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:45.630209 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02af28-37a6-4eac-b83f-f6210651fe87 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:45.630506 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02af28-37a6-4eac-b83f-f6210651fe87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:45.652012 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-3b02af28-37a6-4eac-b83f-f6210651fe87 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:45.653366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-3b02af28-37a6-4eac-b83f-f6210651fe87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:45.728324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1203/2390] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:45.739185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fbc768-1f01-4986-a373-266e01f30015 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:45.740820 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-e7fbc768-1f01-4986-a373-266e01f30015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:45.756854 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:45.812828 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fbc768-1f01-4986-a373-266e01f30015 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:45.817035 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08092be-82c7-4c9b-bd94-85bdd9ef348f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:45.817703 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08092be-82c7-4c9b-bd94-85bdd9ef348f service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:45.817982 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08092be-82c7-4c9b-bd94-85bdd9ef348f service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:45.818254 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08092be-82c7-4c9b-bd94-85bdd9ef348f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:45.829425 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08092be-82c7-4c9b-bd94-85bdd9ef348f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:45.830962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08092be-82c7-4c9b-bd94-85bdd9ef348f service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:45.844363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1188/2391] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:15:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:15:46.343676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02af28-37a6-4eac-b83f-f6210651fe87 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:46.346550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-3b02af28-37a6-4eac-b83f-f6210651fe87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:46.375251 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02af28-37a6-4eac-b83f-f6210651fe87 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:46.382353 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9794947-496f-47e7-bdb5-5b8432fc203a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:46.383059 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9794947-496f-47e7-bdb5-5b8432fc203a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:46.383509 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9794947-496f-47e7-bdb5-5b8432fc203a admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:46.383785 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9794947-496f-47e7-bdb5-5b8432fc203a admin admin] PATH_INFO: `/v3/projects/3300ef591ba14ba0a1d1c437ce613702` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:46.387998 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9794947-496f-47e7-bdb5-5b8432fc203a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3300ef591ba14ba0a1d1c437ce613702)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:46.390119 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9794947-496f-47e7-bdb5-5b8432fc203a admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:46.461397 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e9794947-496f-47e7-bdb5-5b8432fc203a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3300ef591ba14ba0a1d1c437ce613702', 'request_id': 'req-e9794947-496f-47e7-bdb5-5b8432fc203a'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:46.467419 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e9794947-496f-47e7-bdb5-5b8432fc203a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3300ef591ba14ba0a1d1c437ce613702', 'request_id': 'req-e9794947-496f-47e7-bdb5-5b8432fc203a'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:46.474330 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e9794947-496f-47e7-bdb5-5b8432fc203a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3300ef591ba14ba0a1d1c437ce613702', 'request_id': 'req-e9794947-496f-47e7-bdb5-5b8432fc203a'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:46.494126 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e9794947-496f-47e7-bdb5-5b8432fc203a admin admin] The token cache is being invalidate because project 3300ef591ba14ba0a1d1c437ce613702 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:15:46.494471 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e9794947-496f-47e7-bdb5-5b8432fc203a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:46.495310 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e9794947-496f-47e7-bdb5-5b8432fc203a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:46.497292 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1204/2392] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:15:46 2022] DELETE /identity/v3/projects/3300ef591ba14ba0a1d1c437ce613702 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:46.503079 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955a34d-1238-415a-9aae-f11098afe7b1 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:46.503349 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955a34d-1238-415a-9aae-f11098afe7b1 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:46.503714 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955a34d-1238-415a-9aae-f11098afe7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:46.527338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-b955a34d-1238-415a-9aae-f11098afe7b1 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:46.531008 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b955a34d-1238-415a-9aae-f11098afe7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:46.623925 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1189/2393] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:46.632352 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b181c-7de7-49e3-be6e-b41af48db433 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:46.632636 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b181c-7de7-49e3-be6e-b41af48db433 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:46.632907 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b181c-7de7-49e3-be6e-b41af48db433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:46.654334 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-9e5b181c-7de7-49e3-be6e-b41af48db433 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:46.655597 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-9e5b181c-7de7-49e3-be6e-b41af48db433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:46.669180 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1205/2394] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:46.672767 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955a34d-1238-415a-9aae-f11098afe7b1 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:46.684693 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955a34d-1238-415a-9aae-f11098afe7b1 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:46.690542 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3cf91-5e53-4db7-b20d-e3e99e090779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:46.691666 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3cf91-5e53-4db7-b20d-e3e99e090779 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:46.692207 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3cf91-5e53-4db7-b20d-e3e99e090779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:46.692714 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3cf91-5e53-4db7-b20d-e3e99e090779 admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:46.693808 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd3cf91-5e53-4db7-b20d-e3e99e090779 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:15:46.694259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd3cf91-5e53-4db7-b20d-e3e99e090779 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:46.697665 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd3cf91-5e53-4db7-b20d-e3e99e090779 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:46.712854 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1190/2395] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:15:46.729238 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b181c-7de7-49e3-be6e-b41af48db433 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:46.742135 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b181c-7de7-49e3-be6e-b41af48db433 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:46.747672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ef585d-070d-4171-acfc-7f1393b62529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:46.748329 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef585d-070d-4171-acfc-7f1393b62529 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:46.748613 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef585d-070d-4171-acfc-7f1393b62529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:46.748907 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef585d-070d-4171-acfc-7f1393b62529 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:46.749636 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ef585d-070d-4171-acfc-7f1393b62529 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:46.751978 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ef585d-070d-4171-acfc-7f1393b62529 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:46.785816 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1206/2396] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:15:46 2022] POST /identity/v3/projects => generated 361 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:15:46.789484 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3cf91-5e53-4db7-b20d-e3e99e090779 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:46.801953 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3cf91-5e53-4db7-b20d-e3e99e090779 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:46.873844 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b65daf-3e10-46d3-884e-30cc1109fb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:46.874524 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b65daf-3e10-46d3-884e-30cc1109fb50 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:46.874742 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b65daf-3e10-46d3-884e-30cc1109fb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:46.875001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b65daf-3e10-46d3-884e-30cc1109fb50 admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:46.875633 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b65daf-3e10-46d3-884e-30cc1109fb50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:46.877659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b65daf-3e10-46d3-884e-30cc1109fb50 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:46.903129 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1191/2397] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:15:46 2022] POST /identity/v3/users => generated 281 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:15:46.908991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ef585d-070d-4171-acfc-7f1393b62529 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:46.926780 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ef585d-070d-4171-acfc-7f1393b62529 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:46.931707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf4476-4c0a-4490-bcd4-0371e233cb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:46.932334 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf4476-4c0a-4490-bcd4-0371e233cb47 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:46.932636 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf4476-4c0a-4490-bcd4-0371e233cb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:46.932889 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf4476-4c0a-4490-bcd4-0371e233cb47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:46.933590 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf4476-4c0a-4490-bcd4-0371e233cb47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:46.935217 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf4476-4c0a-4490-bcd4-0371e233cb47 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:46.949819 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1207/2398] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:15:46.955014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b65daf-3e10-46d3-884e-30cc1109fb50 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:46.968149 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b65daf-3e10-46d3-884e-30cc1109fb50 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:46.973598 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275f7a8-82af-47a0-9ef5-98abd9a7b083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:46.974422 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275f7a8-82af-47a0-9ef5-98abd9a7b083 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:46.974698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275f7a8-82af-47a0-9ef5-98abd9a7b083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:46.974958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275f7a8-82af-47a0-9ef5-98abd9a7b083 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:46.975607 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1275f7a8-82af-47a0-9ef5-98abd9a7b083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:46.977290 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1275f7a8-82af-47a0-9ef5-98abd9a7b083 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:46.989795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1192/2399] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:15:46.994318 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf4476-4c0a-4490-bcd4-0371e233cb47 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:47.005687 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf4476-4c0a-4490-bcd4-0371e233cb47 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:47.011680 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842da368-e0f2-414b-8c89-8ae2964bbdd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:47.012385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842da368-e0f2-414b-8c89-8ae2964bbdd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:47.012655 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842da368-e0f2-414b-8c89-8ae2964bbdd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:47.012922 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842da368-e0f2-414b-8c89-8ae2964bbdd1 admin admin] PATH_INFO: `/v3/projects/5d53d5317da6421d95356b907d922d3f/users/ca31ab5d7e1b417b970eaa103cfb62cb/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:47.030895 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842da368-e0f2-414b-8c89-8ae2964bbdd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d53d5317da6421d95356b907d922d3f, user_id=ca31ab5d7e1b417b970eaa103cfb62cb, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:47.033878 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842da368-e0f2-414b-8c89-8ae2964bbdd1 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:47.049098 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1208/2400] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:15:46 2022] PUT /identity/v3/projects/5d53d5317da6421d95356b907d922d3f/users/ca31ab5d7e1b417b970eaa103cfb62cb/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:47.062109 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bc3b1-5218-4c10-8e5e-c3e110a57226 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:47.062401 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bc3b1-5218-4c10-8e5e-c3e110a57226 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:47.062663 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bc3b1-5218-4c10-8e5e-c3e110a57226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:47.097035 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-462bc3b1-5218-4c10-8e5e-c3e110a57226 None None] MFA Rules not processed for user `ca31ab5d7e1b417b970eaa103cfb62cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:47.098941 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-462bc3b1-5218-4c10-8e5e-c3e110a57226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:47.112422 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:47.195883 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1193/2401] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:47.239184 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842da368-e0f2-414b-8c89-8ae2964bbdd1 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:47.242013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-842da368-e0f2-414b-8c89-8ae2964bbdd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:47.263230 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:47.333938 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842da368-e0f2-414b-8c89-8ae2964bbdd1 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:47.338263 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219828b9-cade-428e-97b7-9e81ffd8c2f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:47.338887 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219828b9-cade-428e-97b7-9e81ffd8c2f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:47.339161 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219828b9-cade-428e-97b7-9e81ffd8c2f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:47.339426 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219828b9-cade-428e-97b7-9e81ffd8c2f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:47.349452 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219828b9-cade-428e-97b7-9e81ffd8c2f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:47.352145 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219828b9-cade-428e-97b7-9e81ffd8c2f4 service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:47.367407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1209/2402] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:15:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:15:47.385922 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bc3b1-5218-4c10-8e5e-c3e110a57226 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:47.476938 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bc3b1-5218-4c10-8e5e-c3e110a57226 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:47.482143 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde3b79-c197-4fdd-bfe1-5dc43ead550e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:47.482944 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde3b79-c197-4fdd-bfe1-5dc43ead550e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:47.483260 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde3b79-c197-4fdd-bfe1-5dc43ead550e admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:47.483567 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde3b79-c197-4fdd-bfe1-5dc43ead550e admin admin] PATH_INFO: `/v3/users/ca31ab5d7e1b417b970eaa103cfb62cb` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:47.484852 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde3b79-c197-4fdd-bfe1-5dc43ead550e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca31ab5d7e1b417b970eaa103cfb62cb)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:47.487034 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde3b79-c197-4fdd-bfe1-5dc43ead550e admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:47.553969 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8cde3b79-c197-4fdd-bfe1-5dc43ead550e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca31ab5d7e1b417b970eaa103cfb62cb'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:47.564080 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8cde3b79-c197-4fdd-bfe1-5dc43ead550e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca31ab5d7e1b417b970eaa103cfb62cb'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:47.579684 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8cde3b79-c197-4fdd-bfe1-5dc43ead550e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca31ab5d7e1b417b970eaa103cfb62cb'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:47.580613 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8cde3b79-c197-4fdd-bfe1-5dc43ead550e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca31ab5d7e1b417b970eaa103cfb62cb'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:47.589479 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8cde3b79-c197-4fdd-bfe1-5dc43ead550e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca31ab5d7e1b417b970eaa103cfb62cb'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:47.597660 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1194/2403] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:15:47 2022] DELETE /identity/v3/users/ca31ab5d7e1b417b970eaa103cfb62cb => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:47.602587 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7fee6b-584c-4bbb-af49-eb2a2c43ea54 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:47.602873 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7fee6b-584c-4bbb-af49-eb2a2c43ea54 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:47.603160 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7fee6b-584c-4bbb-af49-eb2a2c43ea54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:47.627496 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-fc7fee6b-584c-4bbb-af49-eb2a2c43ea54 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:47.629560 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-fc7fee6b-584c-4bbb-af49-eb2a2c43ea54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:47.742686 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1210/2404] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:47.753390 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde3b79-c197-4fdd-bfe1-5dc43ead550e admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:47.755003 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-8cde3b79-c197-4fdd-bfe1-5dc43ead550e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:47.796521 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:47.826048 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde3b79-c197-4fdd-bfe1-5dc43ead550e admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:47.830786 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344244a8-7a25-418e-85b4-caa4d16f873c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:47.831681 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344244a8-7a25-418e-85b4-caa4d16f873c service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:47.831681 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344244a8-7a25-418e-85b4-caa4d16f873c service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:47.831681 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344244a8-7a25-418e-85b4-caa4d16f873c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:47.843691 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344244a8-7a25-418e-85b4-caa4d16f873c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:47.845228 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344244a8-7a25-418e-85b4-caa4d16f873c service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:47.855599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1195/2405] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:15:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:15:48.174641 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7fee6b-584c-4bbb-af49-eb2a2c43ea54 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:48.176271 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-fc7fee6b-584c-4bbb-af49-eb2a2c43ea54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:48.191148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7fee6b-584c-4bbb-af49-eb2a2c43ea54 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:48.198979 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe9dd76-7842-445f-b29a-735e8035cba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:48.199638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9dd76-7842-445f-b29a-735e8035cba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:48.199915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9dd76-7842-445f-b29a-735e8035cba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:48.200171 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9dd76-7842-445f-b29a-735e8035cba0 admin admin] PATH_INFO: `/v3/projects/5d53d5317da6421d95356b907d922d3f` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:48.201464 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe9dd76-7842-445f-b29a-735e8035cba0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d53d5317da6421d95356b907d922d3f)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:48.203460 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe9dd76-7842-445f-b29a-735e8035cba0 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:48.263103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-afe9dd76-7842-445f-b29a-735e8035cba0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d53d5317da6421d95356b907d922d3f', 'request_id': 'req-afe9dd76-7842-445f-b29a-735e8035cba0'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:48.275389 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-afe9dd76-7842-445f-b29a-735e8035cba0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d53d5317da6421d95356b907d922d3f', 'request_id': 'req-afe9dd76-7842-445f-b29a-735e8035cba0'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:48.284895 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-afe9dd76-7842-445f-b29a-735e8035cba0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d53d5317da6421d95356b907d922d3f', 'request_id': 'req-afe9dd76-7842-445f-b29a-735e8035cba0'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:48.299597 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-afe9dd76-7842-445f-b29a-735e8035cba0 admin admin] The token cache is being invalidate because project 5d53d5317da6421d95356b907d922d3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:15:48.299882 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-afe9dd76-7842-445f-b29a-735e8035cba0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:48.300510 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-afe9dd76-7842-445f-b29a-735e8035cba0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:48.302062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1211/2406] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:15:48 2022] DELETE /identity/v3/projects/5d53d5317da6421d95356b907d922d3f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:48.307224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8286490-e5e2-4d09-bea4-e1dd7c11c332 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:48.307528 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8286490-e5e2-4d09-bea4-e1dd7c11c332 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:48.307798 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8286490-e5e2-4d09-bea4-e1dd7c11c332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:48.337301 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-b8286490-e5e2-4d09-bea4-e1dd7c11c332 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:48.338571 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b8286490-e5e2-4d09-bea4-e1dd7c11c332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:48.407488 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1196/2407] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:48.416993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5a8e4-2972-49f7-ba8a-04fe30b8d0aa None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:48.417696 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5a8e4-2972-49f7-ba8a-04fe30b8d0aa None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:48.417696 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5a8e4-2972-49f7-ba8a-04fe30b8d0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:48.438785 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-a0f5a8e4-2972-49f7-ba8a-04fe30b8d0aa None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:48.440036 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-a0f5a8e4-2972-49f7-ba8a-04fe30b8d0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:48.449912 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1212/2408] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:48.454475 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8286490-e5e2-4d09-bea4-e1dd7c11c332 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:48.469175 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8286490-e5e2-4d09-bea4-e1dd7c11c332 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:48.474775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323e185-1cde-4ea6-9e6f-4da0c383f267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:48.475344 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323e185-1cde-4ea6-9e6f-4da0c383f267 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:48.475619 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323e185-1cde-4ea6-9e6f-4da0c383f267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:48.475888 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323e185-1cde-4ea6-9e6f-4da0c383f267 admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:48.476564 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f323e185-1cde-4ea6-9e6f-4da0c383f267 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:15:48.476837 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f323e185-1cde-4ea6-9e6f-4da0c383f267 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:48.478475 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f323e185-1cde-4ea6-9e6f-4da0c383f267 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:48.495322 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1197/2409] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:15:48.499889 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f5a8e4-2972-49f7-ba8a-04fe30b8d0aa None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:48.512267 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f5a8e4-2972-49f7-ba8a-04fe30b8d0aa None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:48.517180 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2e028-8f18-40a5-910f-470272ede907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:48.517891 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2e028-8f18-40a5-910f-470272ede907 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:48.518163 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2e028-8f18-40a5-910f-470272ede907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:48.518432 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2e028-8f18-40a5-910f-470272ede907 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:48.519099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2e028-8f18-40a5-910f-470272ede907 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:48.521383 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2e028-8f18-40a5-910f-470272ede907 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:48.561386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1213/2410] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:15:48 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:15:48.562889 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323e185-1cde-4ea6-9e6f-4da0c383f267 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:48.576555 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323e185-1cde-4ea6-9e6f-4da0c383f267 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:48.675142 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85f368-e5e6-4c52-9de0-ca3236be829e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:48.675813 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85f368-e5e6-4c52-9de0-ca3236be829e admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:48.676347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85f368-e5e6-4c52-9de0-ca3236be829e admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:48.676607 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85f368-e5e6-4c52-9de0-ca3236be829e admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:48.677847 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab85f368-e5e6-4c52-9de0-ca3236be829e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:48.679528 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab85f368-e5e6-4c52-9de0-ca3236be829e admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:48.697309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97aeff8-3b0a-40f4-864f-7556613b7715 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:48.697732 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97aeff8-3b0a-40f4-864f-7556613b7715 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:48.698122 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97aeff8-3b0a-40f4-864f-7556613b7715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:48.717813 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1198/2411] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:15:48 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:15:48.720170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85f368-e5e6-4c52-9de0-ca3236be829e admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:48.725711 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:48.742675 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85f368-e5e6-4c52-9de0-ca3236be829e admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:48.745425 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-d97aeff8-3b0a-40f4-864f-7556613b7715 None None] MFA Rules not processed for user `3fada78ceaeb4f3bad77c516f6dfcfd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:48.748084 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-d97aeff8-3b0a-40f4-864f-7556613b7715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:48.752609 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf40eee-39dc-4e43-8ee1-a9d278be065d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:48.754144 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf40eee-39dc-4e43-8ee1-a9d278be065d admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:48.755415 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf40eee-39dc-4e43-8ee1-a9d278be065d admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:48.755415 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf40eee-39dc-4e43-8ee1-a9d278be065d admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:48.757549 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf40eee-39dc-4e43-8ee1-a9d278be065d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:48.761522 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf40eee-39dc-4e43-8ee1-a9d278be065d admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:48.783485 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1199/2412] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:15:48.789489 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf40eee-39dc-4e43-8ee1-a9d278be065d admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:48.820577 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf40eee-39dc-4e43-8ee1-a9d278be065d admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:48.836189 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da78a01-ee5f-4fe6-8804-ae207f84707f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:48.837514 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da78a01-ee5f-4fe6-8804-ae207f84707f admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:48.837961 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da78a01-ee5f-4fe6-8804-ae207f84707f admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:48.838496 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da78a01-ee5f-4fe6-8804-ae207f84707f admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:48.839900 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da78a01-ee5f-4fe6-8804-ae207f84707f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:48.842899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da78a01-ee5f-4fe6-8804-ae207f84707f admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:48.862165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1200/2413] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:15:48.869525 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da78a01-ee5f-4fe6-8804-ae207f84707f admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:48.880329 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1214/2414] 10.222.0.44 () {58 vars in 967 bytes} [Sun Sep 11 15:15:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:15:48.887886 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da78a01-ee5f-4fe6-8804-ae207f84707f admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:48.896443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c590d9-12e1-4a82-9e33-5b3b4180aee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:48.898230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c590d9-12e1-4a82-9e33-5b3b4180aee7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:48.899216 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c590d9-12e1-4a82-9e33-5b3b4180aee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:48.899505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c590d9-12e1-4a82-9e33-5b3b4180aee7 admin admin] PATH_INFO: `/v3/projects/79df53c405f04946bcd7687525fb9ea0/users/24c2b3500cd2402c827c6dd66b190aa1/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:48.901888 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97aeff8-3b0a-40f4-864f-7556613b7715 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:48.904109 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-d97aeff8-3b0a-40f4-864f-7556613b7715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:48.927369 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c590d9-12e1-4a82-9e33-5b3b4180aee7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79df53c405f04946bcd7687525fb9ea0, user_id=24c2b3500cd2402c827c6dd66b190aa1, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:48.932782 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c590d9-12e1-4a82-9e33-5b3b4180aee7 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:48.961445 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1201/2415] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:15:48 2022] PUT /identity/v3/projects/79df53c405f04946bcd7687525fb9ea0/users/24c2b3500cd2402c827c6dd66b190aa1/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:49.064874 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893a3f0-7715-44b5-9fc8-50af8be8025f None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:49.065548 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893a3f0-7715-44b5-9fc8-50af8be8025f None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:49.066021 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893a3f0-7715-44b5-9fc8-50af8be8025f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:49.076178 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:49.081987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97aeff8-3b0a-40f4-864f-7556613b7715 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:49.102674 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-0893a3f0-7715-44b5-9fc8-50af8be8025f None None] MFA Rules not processed for user `24c2b3500cd2402c827c6dd66b190aa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:49.104138 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-0893a3f0-7715-44b5-9fc8-50af8be8025f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:49.195295 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f461b-454c-4edf-91c0-f51fe67352ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05aa5e40ee4542719657bc03b2015e72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:49.195943 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f461b-454c-4edf-91c0-f51fe67352ae service swift] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:49.196291 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f461b-454c-4edf-91c0-f51fe67352ae service swift] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:49.196705 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f461b-454c-4edf-91c0-f51fe67352ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:49.229192 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1202/2416] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:15:49.246965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893a3f0-7715-44b5-9fc8-50af8be8025f None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:49.249183 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-0893a3f0-7715-44b5-9fc8-50af8be8025f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:49.300841 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:49.303723 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f461b-454c-4edf-91c0-f51fe67352ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:49.306121 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f461b-454c-4edf-91c0-f51fe67352ae service swift] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:49.315876 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1215/2417] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 15:15:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 15:15:49.332095 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893a3f0-7715-44b5-9fc8-50af8be8025f None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:49.341696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad96f0f-211c-4a60-8659-022addabfe58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:49.342761 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad96f0f-211c-4a60-8659-022addabfe58 service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:49.343092 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad96f0f-211c-4a60-8659-022addabfe58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:49.343462 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad96f0f-211c-4a60-8659-022addabfe58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:49.364169 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad96f0f-211c-4a60-8659-022addabfe58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:49.367305 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad96f0f-211c-4a60-8659-022addabfe58 service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:49.379181 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1203/2418] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:15:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:15:52.246347 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f461b-454c-4edf-91c0-f51fe67352ae service swift] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:52.385307 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f461b-454c-4edf-91c0-f51fe67352ae service swift] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:52.392328 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdbb83-8cca-49cd-a204-0594a9d89f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:52.393296 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdbb83-8cca-49cd-a204-0594a9d89f10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:52.393873 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdbb83-8cca-49cd-a204-0594a9d89f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:52.394485 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdbb83-8cca-49cd-a204-0594a9d89f10 admin admin] PATH_INFO: `/v3/users/24c2b3500cd2402c827c6dd66b190aa1` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:52.395968 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bdbb83-8cca-49cd-a204-0594a9d89f10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24c2b3500cd2402c827c6dd66b190aa1)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:52.398869 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bdbb83-8cca-49cd-a204-0594a9d89f10 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:52.476741 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-70bdbb83-8cca-49cd-a204-0594a9d89f10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24c2b3500cd2402c827c6dd66b190aa1'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:52.488560 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-70bdbb83-8cca-49cd-a204-0594a9d89f10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24c2b3500cd2402c827c6dd66b190aa1'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:52.506784 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-70bdbb83-8cca-49cd-a204-0594a9d89f10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24c2b3500cd2402c827c6dd66b190aa1'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:52.507647 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-70bdbb83-8cca-49cd-a204-0594a9d89f10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24c2b3500cd2402c827c6dd66b190aa1'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:52.524374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-70bdbb83-8cca-49cd-a204-0594a9d89f10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24c2b3500cd2402c827c6dd66b190aa1'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:52.537602 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1216/2419] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:15:52 2022] DELETE /identity/v3/users/24c2b3500cd2402c827c6dd66b190aa1 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:15:52.543017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17fb6f-eab9-457c-a32f-1aacd0050053 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:52.543865 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17fb6f-eab9-457c-a32f-1aacd0050053 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:52.543865 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17fb6f-eab9-457c-a32f-1aacd0050053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:52.593919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-3d17fb6f-eab9-457c-a32f-1aacd0050053 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:15:52.594658 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-3d17fb6f-eab9-457c-a32f-1aacd0050053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:52.688669 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1204/2420] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:15:52.697007 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdbb83-8cca-49cd-a204-0594a9d89f10 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:52.700099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-70bdbb83-8cca-49cd-a204-0594a9d89f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:52.812708 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdbb83-8cca-49cd-a204-0594a9d89f10 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:52.819260 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82268b94-fc10-4c84-b237-f9eb3684ad21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:52.821216 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82268b94-fc10-4c84-b237-f9eb3684ad21 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:52.834872 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82268b94-fc10-4c84-b237-f9eb3684ad21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:52.835214 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82268b94-fc10-4c84-b237-f9eb3684ad21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:52.856463 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82268b94-fc10-4c84-b237-f9eb3684ad21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:52.859875 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82268b94-fc10-4c84-b237-f9eb3684ad21 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:52.893342 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1217/2421] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:15:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:15:53.407954 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d17fb6f-eab9-457c-a32f-1aacd0050053 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:15:53.410131 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-3d17fb6f-eab9-457c-a32f-1aacd0050053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:15:53.422833 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d17fb6f-eab9-457c-a32f-1aacd0050053 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:15:53.428714 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07e890-b7f7-4f01-9e37-e2bc6c6cf7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:15:53.429598 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07e890-b7f7-4f01-9e37-e2bc6c6cf7ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:15:53.429969 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07e890-b7f7-4f01-9e37-e2bc6c6cf7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:15:53.430798 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07e890-b7f7-4f01-9e37-e2bc6c6cf7ea admin admin] PATH_INFO: `/v3/projects/79df53c405f04946bcd7687525fb9ea0` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:15:53.432154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07e890-b7f7-4f01-9e37-e2bc6c6cf7ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=79df53c405f04946bcd7687525fb9ea0)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:15:53.434310 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07e890-b7f7-4f01-9e37-e2bc6c6cf7ea admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:15:53.488470 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:15:53.517403 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8d07e890-b7f7-4f01-9e37-e2bc6c6cf7ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79df53c405f04946bcd7687525fb9ea0', 'request_id': 'req-8d07e890-b7f7-4f01-9e37-e2bc6c6cf7ea'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:53.531360 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8d07e890-b7f7-4f01-9e37-e2bc6c6cf7ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79df53c405f04946bcd7687525fb9ea0', 'request_id': 'req-8d07e890-b7f7-4f01-9e37-e2bc6c6cf7ea'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:53.539863 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8d07e890-b7f7-4f01-9e37-e2bc6c6cf7ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79df53c405f04946bcd7687525fb9ea0', 'request_id': 'req-8d07e890-b7f7-4f01-9e37-e2bc6c6cf7ea'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:53.566392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8d07e890-b7f7-4f01-9e37-e2bc6c6cf7ea admin admin] The token cache is being invalidate because project 79df53c405f04946bcd7687525fb9ea0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:15:53.567456 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8d07e890-b7f7-4f01-9e37-e2bc6c6cf7ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:53.570300 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8d07e890-b7f7-4f01-9e37-e2bc6c6cf7ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:15:53.573067 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1205/2422] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:15:53 2022] DELETE /identity/v3/projects/79df53c405f04946bcd7687525fb9ea0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:16:02.770958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c03e8b-3997-4384-8222-5058a4b33145 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:16:02.771603 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c03e8b-3997-4384-8222-5058a4b33145 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:16:02.771603 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c03e8b-3997-4384-8222-5058a4b33145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:16:02.792141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-15c03e8b-3997-4384-8222-5058a4b33145 None None] MFA Rules not processed for user `c44105f2d90e469fb03f311a4cf1476f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:16:02.793397 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-15c03e8b-3997-4384-8222-5058a4b33145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:16:02.806938 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:16:02.879880 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1218/2423] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:16:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:16:03.553898 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:16:12.871603 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=17 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:16:34.688695 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b155a1-39cc-4eab-b5e8-72dda45db789 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:16:34.689504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b155a1-39cc-4eab-b5e8-72dda45db789 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:16:34.689504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b155a1-39cc-4eab-b5e8-72dda45db789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:16:34.709534 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-b2b155a1-39cc-4eab-b5e8-72dda45db789 None None] MFA Rules not processed for user `0292da3873ef419097717fd11337a128`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:16:34.710763 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b2b155a1-39cc-4eab-b5e8-72dda45db789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:16:34.779169 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1206/2424] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:16:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:16:39.089026 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e49b5e-d27f-460a-ace5-e296af1eab81 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:16:39.089626 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e49b5e-d27f-460a-ace5-e296af1eab81 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:16:39.089626 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e49b5e-d27f-460a-ace5-e296af1eab81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:16:39.110832 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-87e49b5e-d27f-460a-ace5-e296af1eab81 None None] MFA Rules not processed for user `2bd47bf9aa4142b1b7ec8d02e8661578`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:16:39.112050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-87e49b5e-d27f-460a-ace5-e296af1eab81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:16:39.176105 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1219/2425] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:16:39 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:16:44.772304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=25 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:16:46.852816 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811430f-4a95-40c2-a73c-afa55ae0b40d None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:16:46.853523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811430f-4a95-40c2-a73c-afa55ae0b40d None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:16:46.853523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811430f-4a95-40c2-a73c-afa55ae0b40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:16:46.882795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-0811430f-4a95-40c2-a73c-afa55ae0b40d None None] MFA Rules not processed for user `5e4f8b13b66449cf849ff9cb73938c4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:16:46.885516 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-0811430f-4a95-40c2-a73c-afa55ae0b40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:16:46.951850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1207/2426] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:16:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:16:49.168617 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=25 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:16:56.943811 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=25 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:17:20.543722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e49b5e-d27f-460a-ace5-e296af1eab81 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:17:20.545524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-87e49b5e-d27f-460a-ace5-e296af1eab81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:17:20.611356 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e49b5e-d27f-460a-ace5-e296af1eab81 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:17:20.615220 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050564e3-5b01-4671-99ef-31b4e84e61da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:17:20.615838 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050564e3-5b01-4671-99ef-31b4e84e61da service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:17:20.616114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050564e3-5b01-4671-99ef-31b4e84e61da service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:17:20.616380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050564e3-5b01-4671-99ef-31b4e84e61da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:17:20.618267 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-050564e3-5b01-4671-99ef-31b4e84e61da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:17:20.669044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811430f-4a95-40c2-a73c-afa55ae0b40d None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:17:20.680210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811430f-4a95-40c2-a73c-afa55ae0b40d None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:17:20.686967 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea7dc5-9f4b-4012-9ff1-9afc89ac12e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:17:20.687882 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea7dc5-9f4b-4012-9ff1-9afc89ac12e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:17:20.688707 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea7dc5-9f4b-4012-9ff1-9afc89ac12e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:17:20.689093 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea7dc5-9f4b-4012-9ff1-9afc89ac12e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:17:20.692891 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-a6ea7dc5-9f4b-4012-9ff1-9afc89ac12e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:17:20.712150 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050564e3-5b01-4671-99ef-31b4e84e61da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:17:20.715276 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050564e3-5b01-4671-99ef-31b4e84e61da service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:17:20.738942 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1220/2427] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:17:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:17:20.810019 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea7dc5-9f4b-4012-9ff1-9afc89ac12e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:17:20.811520 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea7dc5-9f4b-4012-9ff1-9afc89ac12e1 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:17:20.824824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1208/2428] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:17:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:17:25.462593 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c7e27-7460-4910-8ee6-e6bcd1e727e6 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:17:25.463261 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c7e27-7460-4910-8ee6-e6bcd1e727e6 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:17:25.463261 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c7e27-7460-4910-8ee6-e6bcd1e727e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:17:25.483137 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-ba2c7e27-7460-4910-8ee6-e6bcd1e727e6 None None] MFA Rules not processed for user `c44105f2d90e469fb03f311a4cf1476f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:17:25.484628 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-ba2c7e27-7460-4910-8ee6-e6bcd1e727e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:17:25.492008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1221/2429] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:17:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:17:30.818946 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=34 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:17:35.477229 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=57 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:17:38.377984 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea7dc5-9f4b-4012-9ff1-9afc89ac12e1 service neutron] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:17:38.380033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-a6ea7dc5-9f4b-4012-9ff1-9afc89ac12e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:17:38.403299 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea7dc5-9f4b-4012-9ff1-9afc89ac12e1 service neutron] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:17:38.411979 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61da07-a489-435d-9a3e-5eb34b115984 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:17:38.412903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61da07-a489-435d-9a3e-5eb34b115984 service nova] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:17:38.413405 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61da07-a489-435d-9a3e-5eb34b115984 service nova] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:17:38.413852 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61da07-a489-435d-9a3e-5eb34b115984 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:17:38.416877 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-1e61da07-a489-435d-9a3e-5eb34b115984 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:17:38.432428 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e61da07-a489-435d-9a3e-5eb34b115984 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:17:38.436205 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e61da07-a489-435d-9a3e-5eb34b115984 service nova] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:17:38.455739 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1209/2430] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:17:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:17:41.077964 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c7e27-7460-4910-8ee6-e6bcd1e727e6 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:17:41.080550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-ba2c7e27-7460-4910-8ee6-e6bcd1e727e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:17:41.168224 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c7e27-7460-4910-8ee6-e6bcd1e727e6 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:17:41.172710 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717ad18-7146-46fe-9bde-93366426b57f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:17:41.173404 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717ad18-7146-46fe-9bde-93366426b57f service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:17:41.173677 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717ad18-7146-46fe-9bde-93366426b57f service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:17:41.175006 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717ad18-7146-46fe-9bde-93366426b57f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:17:41.178351 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-6717ad18-7146-46fe-9bde-93366426b57f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:17:41.201747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6717ad18-7146-46fe-9bde-93366426b57f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:17:41.204373 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6717ad18-7146-46fe-9bde-93366426b57f service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:17:41.231335 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1222/2431] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:17:41 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:17:46.365295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61da07-a489-435d-9a3e-5eb34b115984 service nova] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:17:46.367695 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-1e61da07-a489-435d-9a3e-5eb34b115984 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:17:46.452248 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717ad18-7146-46fe-9bde-93366426b57f service cinder] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:17:46.485774 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717ad18-7146-46fe-9bde-93366426b57f service cinder] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:17:46.496336 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70cfcd-0232-4fc6-ad85-63cd1141c7ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:17:46.497562 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70cfcd-0232-4fc6-ad85-63cd1141c7ce service nova] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:17:46.498262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70cfcd-0232-4fc6-ad85-63cd1141c7ce service nova] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:17:46.498971 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70cfcd-0232-4fc6-ad85-63cd1141c7ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:17:46.502522 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61da07-a489-435d-9a3e-5eb34b115984 service nova] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:17:46.502875 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-bb70cfcd-0232-4fc6-ad85-63cd1141c7ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:17:46.509024 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b8413-62fa-4802-b461-99885b813f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:17:46.510109 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b8413-62fa-4802-b461-99885b813f16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:17:46.510579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b8413-62fa-4802-b461-99885b813f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:17:46.511063 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b8413-62fa-4802-b461-99885b813f16 admin admin] PATH_INFO: `/v3/users/0292da3873ef419097717fd11337a128` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:17:46.512913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901b8413-62fa-4802-b461-99885b813f16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0292da3873ef419097717fd11337a128)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:17:46.515375 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901b8413-62fa-4802-b461-99885b813f16 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:17:46.529481 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb70cfcd-0232-4fc6-ad85-63cd1141c7ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:17:46.532743 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb70cfcd-0232-4fc6-ad85-63cd1141c7ce service nova] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:17:46.557437 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1223/2432] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:17:46 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:17:46.610116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-901b8413-62fa-4802-b461-99885b813f16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0292da3873ef419097717fd11337a128'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:17:46.626797 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-901b8413-62fa-4802-b461-99885b813f16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0292da3873ef419097717fd11337a128'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:17:46.656600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-901b8413-62fa-4802-b461-99885b813f16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0292da3873ef419097717fd11337a128'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:17:46.658080 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-901b8413-62fa-4802-b461-99885b813f16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0292da3873ef419097717fd11337a128'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:17:46.675454 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-901b8413-62fa-4802-b461-99885b813f16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0292da3873ef419097717fd11337a128'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:17:46.693458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1210/2433] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:17:46 2022] DELETE /identity/v3/users/0292da3873ef419097717fd11337a128 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:17:47.228598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70cfcd-0232-4fc6-ad85-63cd1141c7ce service nova] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:17:47.243351 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70cfcd-0232-4fc6-ad85-63cd1141c7ce service nova] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:17:47.338999 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318c1cb-5a43-47b4-8fa1-60ac89383a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:17:47.339745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318c1cb-5a43-47b4-8fa1-60ac89383a38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:17:47.340028 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318c1cb-5a43-47b4-8fa1-60ac89383a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:17:47.340296 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318c1cb-5a43-47b4-8fa1-60ac89383a38 admin admin] PATH_INFO: `/v3/projects/fd7984f5434f42098fb127572b6d1e0c` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:17:47.342289 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5318c1cb-5a43-47b4-8fa1-60ac89383a38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd7984f5434f42098fb127572b6d1e0c)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:17:47.344245 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5318c1cb-5a43-47b4-8fa1-60ac89383a38 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:17:47.411102 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5318c1cb-5a43-47b4-8fa1-60ac89383a38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd7984f5434f42098fb127572b6d1e0c', 'request_id': 'req-5318c1cb-5a43-47b4-8fa1-60ac89383a38'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:17:47.422113 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5318c1cb-5a43-47b4-8fa1-60ac89383a38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd7984f5434f42098fb127572b6d1e0c', 'request_id': 'req-5318c1cb-5a43-47b4-8fa1-60ac89383a38'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:17:47.429776 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5318c1cb-5a43-47b4-8fa1-60ac89383a38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd7984f5434f42098fb127572b6d1e0c', 'request_id': 'req-5318c1cb-5a43-47b4-8fa1-60ac89383a38'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:17:47.446168 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5318c1cb-5a43-47b4-8fa1-60ac89383a38 admin admin] The token cache is being invalidate because project fd7984f5434f42098fb127572b6d1e0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:17:47.446457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5318c1cb-5a43-47b4-8fa1-60ac89383a38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:17:47.447219 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5318c1cb-5a43-47b4-8fa1-60ac89383a38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:17:47.448731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1224/2434] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:17:47 2022] DELETE /identity/v3/projects/fd7984f5434f42098fb127572b6d1e0c => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:17:49.227046 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b8413-62fa-4802-b461-99885b813f16 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:17:49.228767 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-901b8413-62fa-4802-b461-99885b813f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:17:49.295213 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b8413-62fa-4802-b461-99885b813f16 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:17:49.314808 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:17:49.355389 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975f515-95f0-4780-8af9-ac6afbc758be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:17:49.356392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975f515-95f0-4780-8af9-ac6afbc758be service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:17:49.357032 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975f515-95f0-4780-8af9-ac6afbc758be service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:17:49.357517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975f515-95f0-4780-8af9-ac6afbc758be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:17:49.360350 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-2975f515-95f0-4780-8af9-ac6afbc758be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:17:49.375369 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975f515-95f0-4780-8af9-ac6afbc758be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:17:49.376882 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975f515-95f0-4780-8af9-ac6afbc758be service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:17:49.386741 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1211/2435] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:17:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:17:51.708286 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318c1cb-5a43-47b4-8fa1-60ac89383a38 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:17:51.709772 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-5318c1cb-5a43-47b4-8fa1-60ac89383a38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:17:51.723016 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318c1cb-5a43-47b4-8fa1-60ac89383a38 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:17:51.726540 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248061a-bff9-49b3-91a8-be5693ac4dff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:17:51.727143 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248061a-bff9-49b3-91a8-be5693ac4dff service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:17:51.727414 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248061a-bff9-49b3-91a8-be5693ac4dff service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:17:51.727719 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248061a-bff9-49b3-91a8-be5693ac4dff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:17:51.729600 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-c248061a-bff9-49b3-91a8-be5693ac4dff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:17:51.749317 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:17:51.805879 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248061a-bff9-49b3-91a8-be5693ac4dff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:17:51.807351 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248061a-bff9-49b3-91a8-be5693ac4dff service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:17:51.819486 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1225/2436] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:17:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:17:55.636365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975f515-95f0-4780-8af9-ac6afbc758be service neutron] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:17:55.639363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-2975f515-95f0-4780-8af9-ac6afbc758be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:17:55.665244 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975f515-95f0-4780-8af9-ac6afbc758be service neutron] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:17:55.669920 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358ee59-6e14-45d4-958e-aa2d6cfe6eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:17:55.670564 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358ee59-6e14-45d4-958e-aa2d6cfe6eb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:17:55.670840 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358ee59-6e14-45d4-958e-aa2d6cfe6eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:17:55.671111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358ee59-6e14-45d4-958e-aa2d6cfe6eb4 admin admin] PATH_INFO: `/v3/users/2bd47bf9aa4142b1b7ec8d02e8661578` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:17:55.672295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358ee59-6e14-45d4-958e-aa2d6cfe6eb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bd47bf9aa4142b1b7ec8d02e8661578)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:17:55.674278 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358ee59-6e14-45d4-958e-aa2d6cfe6eb4 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:17:55.738398 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3358ee59-6e14-45d4-958e-aa2d6cfe6eb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bd47bf9aa4142b1b7ec8d02e8661578'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:17:55.747184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3358ee59-6e14-45d4-958e-aa2d6cfe6eb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bd47bf9aa4142b1b7ec8d02e8661578'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:17:55.760283 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3358ee59-6e14-45d4-958e-aa2d6cfe6eb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bd47bf9aa4142b1b7ec8d02e8661578'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:17:55.761067 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3358ee59-6e14-45d4-958e-aa2d6cfe6eb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bd47bf9aa4142b1b7ec8d02e8661578'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:17:55.768152 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3358ee59-6e14-45d4-958e-aa2d6cfe6eb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bd47bf9aa4142b1b7ec8d02e8661578'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:17:55.774667 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1212/2437] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:17:55 2022] DELETE /identity/v3/users/2bd47bf9aa4142b1b7ec8d02e8661578 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:17:55.971366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248061a-bff9-49b3-91a8-be5693ac4dff service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:17:55.983078 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248061a-bff9-49b3-91a8-be5693ac4dff service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:17:56.049532 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac335126-46e0-4f19-ba9f-b0c8642106f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:17:56.050324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac335126-46e0-4f19-ba9f-b0c8642106f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:17:56.050602 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac335126-46e0-4f19-ba9f-b0c8642106f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:17:56.050870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac335126-46e0-4f19-ba9f-b0c8642106f6 admin admin] PATH_INFO: `/v3/projects/7933806a47704d2d867be7312890b538` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:17:56.052074 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac335126-46e0-4f19-ba9f-b0c8642106f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7933806a47704d2d867be7312890b538)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:17:56.054037 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac335126-46e0-4f19-ba9f-b0c8642106f6 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:17:56.110636 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-ac335126-46e0-4f19-ba9f-b0c8642106f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7933806a47704d2d867be7312890b538', 'request_id': 'req-ac335126-46e0-4f19-ba9f-b0c8642106f6'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:17:56.116319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-ac335126-46e0-4f19-ba9f-b0c8642106f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7933806a47704d2d867be7312890b538', 'request_id': 'req-ac335126-46e0-4f19-ba9f-b0c8642106f6'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:17:56.122951 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-ac335126-46e0-4f19-ba9f-b0c8642106f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7933806a47704d2d867be7312890b538', 'request_id': 'req-ac335126-46e0-4f19-ba9f-b0c8642106f6'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:17:56.139204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-ac335126-46e0-4f19-ba9f-b0c8642106f6 admin admin] The token cache is being invalidate because project 7933806a47704d2d867be7312890b538 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:17:56.139495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-ac335126-46e0-4f19-ba9f-b0c8642106f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:17:56.140207 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-ac335126-46e0-4f19-ba9f-b0c8642106f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:17:56.141755 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1226/2438] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:17:55 2022] DELETE /identity/v3/projects/7933806a47704d2d867be7312890b538 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:17:56.147088 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0602045-ef67-4a8e-9b0d-cc00ee293b03 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:17:56.147362 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0602045-ef67-4a8e-9b0d-cc00ee293b03 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:17:56.147654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0602045-ef67-4a8e-9b0d-cc00ee293b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:17:56.169523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-b0602045-ef67-4a8e-9b0d-cc00ee293b03 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:17:56.170775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b0602045-ef67-4a8e-9b0d-cc00ee293b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:17:56.243574 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1213/2439] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:17:56.252312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24af59-03ff-4680-b962-6f73ba21dabf None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:17:56.252560 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24af59-03ff-4680-b962-6f73ba21dabf None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:17:56.253181 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24af59-03ff-4680-b962-6f73ba21dabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:17:56.275395 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-3a24af59-03ff-4680-b962-6f73ba21dabf None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:17:56.276691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-3a24af59-03ff-4680-b962-6f73ba21dabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:17:56.286935 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1227/2440] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:17:56.291600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0602045-ef67-4a8e-9b0d-cc00ee293b03 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:17:56.301774 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0602045-ef67-4a8e-9b0d-cc00ee293b03 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:17:56.306970 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc5f32-0fdc-45be-b09f-cd2334087deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:17:56.307691 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc5f32-0fdc-45be-b09f-cd2334087deb admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:17:56.307968 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc5f32-0fdc-45be-b09f-cd2334087deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:17:56.308230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc5f32-0fdc-45be-b09f-cd2334087deb admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:17:56.308974 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc5f32-0fdc-45be-b09f-cd2334087deb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:17:56.309307 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc5f32-0fdc-45be-b09f-cd2334087deb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:17:56.310966 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc5f32-0fdc-45be-b09f-cd2334087deb admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:17:56.327720 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1214/2441] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:17:56.329015 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a24af59-03ff-4680-b962-6f73ba21dabf None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:17:56.339781 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a24af59-03ff-4680-b962-6f73ba21dabf None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:17:56.344789 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97622ae9-253a-4a2b-9fda-6f4c68032170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:17:56.345468 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97622ae9-253a-4a2b-9fda-6f4c68032170 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:17:56.345716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97622ae9-253a-4a2b-9fda-6f4c68032170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:17:56.345998 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97622ae9-253a-4a2b-9fda-6f4c68032170 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:17:56.346675 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97622ae9-253a-4a2b-9fda-6f4c68032170 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:17:56.348555 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97622ae9-253a-4a2b-9fda-6f4c68032170 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:17:56.379745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1228/2442] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:17:56 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:17:56.383745 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc5f32-0fdc-45be-b09f-cd2334087deb admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:17:56.398032 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc5f32-0fdc-45be-b09f-cd2334087deb admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:17:56.422570 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:17:56.464512 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8f187-9695-44c0-980d-bac4ee0be1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:17:56.465117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8f187-9695-44c0-980d-bac4ee0be1bf admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:17:56.465420 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8f187-9695-44c0-980d-bac4ee0be1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:17:56.465683 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8f187-9695-44c0-980d-bac4ee0be1bf admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:17:56.466309 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f8f187-9695-44c0-980d-bac4ee0be1bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:17:56.468218 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f8f187-9695-44c0-980d-bac4ee0be1bf admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:17:56.495128 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1215/2443] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:17:56 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:17:56.505048 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97622ae9-253a-4a2b-9fda-6f4c68032170 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:17:56.516502 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97622ae9-253a-4a2b-9fda-6f4c68032170 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:17:56.521898 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7882c70-8272-431a-b9a4-bc475443ef4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:17:56.529119 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7882c70-8272-431a-b9a4-bc475443ef4a admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:17:56.529119 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7882c70-8272-431a-b9a4-bc475443ef4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:17:56.529119 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7882c70-8272-431a-b9a4-bc475443ef4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:17:56.529119 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7882c70-8272-431a-b9a4-bc475443ef4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:17:56.529119 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7882c70-8272-431a-b9a4-bc475443ef4a admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:17:56.537292 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1229/2444] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:17:56.541719 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8f187-9695-44c0-980d-bac4ee0be1bf admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:17:56.552779 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8f187-9695-44c0-980d-bac4ee0be1bf admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:17:56.557893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a3182-ef28-42f9-9ea2-a13ab7a63311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:17:56.558520 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a3182-ef28-42f9-9ea2-a13ab7a63311 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:17:56.558790 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a3182-ef28-42f9-9ea2-a13ab7a63311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:17:56.559065 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a3182-ef28-42f9-9ea2-a13ab7a63311 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:17:56.559715 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a3182-ef28-42f9-9ea2-a13ab7a63311 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:17:56.561384 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a3182-ef28-42f9-9ea2-a13ab7a63311 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:17:56.574133 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1216/2445] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:17:56.578813 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7882c70-8272-431a-b9a4-bc475443ef4a admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:17:56.585538 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:17:56.589534 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7882c70-8272-431a-b9a4-bc475443ef4a admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:17:56.595564 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78897cb7-f72d-43ce-bd5e-192a0f3b203a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:17:56.596252 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78897cb7-f72d-43ce-bd5e-192a0f3b203a admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:17:56.596790 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78897cb7-f72d-43ce-bd5e-192a0f3b203a admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:17:56.597119 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78897cb7-f72d-43ce-bd5e-192a0f3b203a admin admin] PATH_INFO: `/v3/projects/bf1611f722d34ce0b058da277806b326/users/948640729c5c4975b3b050129eb63eea/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:17:56.615296 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78897cb7-f72d-43ce-bd5e-192a0f3b203a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf1611f722d34ce0b058da277806b326, user_id=948640729c5c4975b3b050129eb63eea, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:17:56.618157 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78897cb7-f72d-43ce-bd5e-192a0f3b203a admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:17:56.633319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1230/2446] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:17:56 2022] PUT /identity/v3/projects/bf1611f722d34ce0b058da277806b326/users/948640729c5c4975b3b050129eb63eea/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:17:56.638737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4773e8-f6ef-4550-bf3a-dc33932df3d6 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:17:56.639353 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4773e8-f6ef-4550-bf3a-dc33932df3d6 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:17:56.639450 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4773e8-f6ef-4550-bf3a-dc33932df3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:17:56.660400 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-5b4773e8-f6ef-4550-bf3a-dc33932df3d6 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:17:56.661659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-5b4773e8-f6ef-4550-bf3a-dc33932df3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:17:56.725744 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1217/2447] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:17:56.737568 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78897cb7-f72d-43ce-bd5e-192a0f3b203a admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:17:56.739779 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-78897cb7-f72d-43ce-bd5e-192a0f3b203a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:17:56.812430 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78897cb7-f72d-43ce-bd5e-192a0f3b203a admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:17:56.818576 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c3300d-6fc2-411d-a49b-521620bcbb39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:17:56.819360 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3300d-6fc2-411d-a49b-521620bcbb39 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:17:56.819724 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3300d-6fc2-411d-a49b-521620bcbb39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:17:56.820072 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3300d-6fc2-411d-a49b-521620bcbb39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:17:56.831010 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c3300d-6fc2-411d-a49b-521620bcbb39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:17:56.832474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c3300d-6fc2-411d-a49b-521620bcbb39 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:17:56.846790 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1231/2448] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:17:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:18:01.439824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d34ce4-5ba6-495c-bff3-ecb47c2de541 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:01.440384 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d34ce4-5ba6-495c-bff3-ecb47c2de541 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:01.440384 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d34ce4-5ba6-495c-bff3-ecb47c2de541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:01.482039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-62d34ce4-5ba6-495c-bff3-ecb47c2de541 None None] MFA Rules not processed for user `948640729c5c4975b3b050129eb63eea`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:18:01.483528 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-62d34ce4-5ba6-495c-bff3-ecb47c2de541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:18:01.576676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1218/2449] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:18:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:18:01.592162 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c3300d-6fc2-411d-a49b-521620bcbb39 service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:01.594705 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-83c3300d-6fc2-411d-a49b-521620bcbb39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:18:01.696750 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c3300d-6fc2-411d-a49b-521620bcbb39 service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:01.702557 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d6482-0822-4cd9-8d42-30dc5e69ca92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:01.703182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d6482-0822-4cd9-8d42-30dc5e69ca92 service nova] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:01.703463 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d6482-0822-4cd9-8d42-30dc5e69ca92 service nova] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:01.703734 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d6482-0822-4cd9-8d42-30dc5e69ca92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:01.712374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d6482-0822-4cd9-8d42-30dc5e69ca92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:18:01.714009 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d6482-0822-4cd9-8d42-30dc5e69ca92 service nova] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:18:01.723860 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1232/2450] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:18:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:18:03.205676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8fa02-e9a5-429d-ab67-df0a3e250d79 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:03.206157 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8fa02-e9a5-429d-ab67-df0a3e250d79 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:03.206253 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8fa02-e9a5-429d-ab67-df0a3e250d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:03.233242 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-a7e8fa02-e9a5-429d-ab67-df0a3e250d79 None None] MFA Rules not processed for user `948640729c5c4975b3b050129eb63eea`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:18:03.234688 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-a7e8fa02-e9a5-429d-ab67-df0a3e250d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:18:03.243045 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1219/2451] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:18:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:18:03.593200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d6482-0822-4cd9-8d42-30dc5e69ca92 service nova] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:03.605624 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d6482-0822-4cd9-8d42-30dc5e69ca92 service nova] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:03.610275 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c808db-f7ba-4d13-b378-b262d6fd74e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:03.610921 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c808db-f7ba-4d13-b378-b262d6fd74e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:03.611206 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c808db-f7ba-4d13-b378-b262d6fd74e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:03.611478 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c808db-f7ba-4d13-b378-b262d6fd74e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:03.613437 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-00c808db-f7ba-4d13-b378-b262d6fd74e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:18:03.624528 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c808db-f7ba-4d13-b378-b262d6fd74e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:18:03.625959 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c808db-f7ba-4d13-b378-b262d6fd74e2 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:18:03.637978 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1233/2452] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:18:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:18:04.069644 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8fa02-e9a5-429d-ab67-df0a3e250d79 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:04.071787 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-a7e8fa02-e9a5-429d-ab67-df0a3e250d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:18:04.132489 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:18:04.188919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8fa02-e9a5-429d-ab67-df0a3e250d79 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:04.194821 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d9f6b-2cf2-4ac7-9acc-bf163c60cb25 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead06362f8a2444bbfe1405cd602f1c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:04.195448 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d9f6b-2cf2-4ac7-9acc-bf163c60cb25 service placement] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:04.195729 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d9f6b-2cf2-4ac7-9acc-bf163c60cb25 service placement] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:04.195995 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d9f6b-2cf2-4ac7-9acc-bf163c60cb25 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:04.197975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-e18d9f6b-2cf2-4ac7-9acc-bf163c60cb25 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:18:04.211579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d9f6b-2cf2-4ac7-9acc-bf163c60cb25 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:18:04.213489 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d9f6b-2cf2-4ac7-9acc-bf163c60cb25 service placement] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:18:04.225240 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1220/2453] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:18:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:18:07.287721 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c808db-f7ba-4d13-b378-b262d6fd74e2 service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:07.302894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c808db-f7ba-4d13-b378-b262d6fd74e2 service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:07.309413 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8fbf38-0e2e-48eb-9c0a-bb7ab9324a20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:07.310293 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fbf38-0e2e-48eb-9c0a-bb7ab9324a20 service nova] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:07.310551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fbf38-0e2e-48eb-9c0a-bb7ab9324a20 service nova] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:07.311086 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fbf38-0e2e-48eb-9c0a-bb7ab9324a20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:07.312934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-7e8fbf38-0e2e-48eb-9c0a-bb7ab9324a20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:18:07.326050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8fbf38-0e2e-48eb-9c0a-bb7ab9324a20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:18:07.327663 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8fbf38-0e2e-48eb-9c0a-bb7ab9324a20 service nova] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:18:07.338955 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1234/2454] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:18:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:18:12.081548 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5b4c8-f6b6-4c4f-a89d-d24ae1a563e5 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:12.082127 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5b4c8-f6b6-4c4f-a89d-d24ae1a563e5 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:12.082127 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5b4c8-f6b6-4c4f-a89d-d24ae1a563e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:12.103156 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-f0b5b4c8-f6b6-4c4f-a89d-d24ae1a563e5 None None] MFA Rules not processed for user `3fada78ceaeb4f3bad77c516f6dfcfd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:18:12.104395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f0b5b4c8-f6b6-4c4f-a89d-d24ae1a563e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:18:12.160476 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1221/2455] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 15:18:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:18:12.211581 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8fbf38-0e2e-48eb-9c0a-bb7ab9324a20 service nova] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:12.213295 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-7e8fbf38-0e2e-48eb-9c0a-bb7ab9324a20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:18:12.265418 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:18:12.296053 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8fbf38-0e2e-48eb-9c0a-bb7ab9324a20 service nova] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:12.301580 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25285ed2-16ca-4089-b089-71c38f465dbb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05aa5e40ee4542719657bc03b2015e72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:12.302251 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25285ed2-16ca-4089-b089-71c38f465dbb service swift] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:12.302524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25285ed2-16ca-4089-b089-71c38f465dbb service swift] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:12.302794 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25285ed2-16ca-4089-b089-71c38f465dbb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:12.312075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25285ed2-16ca-4089-b089-71c38f465dbb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:18:12.313699 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25285ed2-16ca-4089-b089-71c38f465dbb service swift] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:18:12.323632 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1235/2456] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 15:18:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 15:18:13.019654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b5b4c8-f6b6-4c4f-a89d-d24ae1a563e5 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:13.022031 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f0b5b4c8-f6b6-4c4f-a89d-d24ae1a563e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:18:13.047923 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b5b4c8-f6b6-4c4f-a89d-d24ae1a563e5 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:13.055344 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aca55e-d68c-40e6-a9ae-61ee26898639 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:13.055990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aca55e-d68c-40e6-a9ae-61ee26898639 service nova] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:13.056486 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aca55e-d68c-40e6-a9ae-61ee26898639 service nova] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:13.056808 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aca55e-d68c-40e6-a9ae-61ee26898639 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:13.058726 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f2aca55e-d68c-40e6-a9ae-61ee26898639 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:18:13.082065 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aca55e-d68c-40e6-a9ae-61ee26898639 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:18:13.084901 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aca55e-d68c-40e6-a9ae-61ee26898639 service nova] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:18:13.112085 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1222/2457] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:18:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:18:22.319167 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=13 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:18:23.092964 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=51 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:18:26.714398 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b003d5d-d99a-4ac6-b958-f895220ff7c9 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:26.715486 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b003d5d-d99a-4ac6-b958-f895220ff7c9 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:26.716203 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b003d5d-d99a-4ac6-b958-f895220ff7c9 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:26.719849 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1236/2458] 10.222.0.44 () {54 vars in 921 bytes} [Sun Sep 11 15:18:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:18:26.726628 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c4f1c-2def-4ab8-ae99-8e0ea3a32e23 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:26.727535 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c4f1c-2def-4ab8-ae99-8e0ea3a32e23 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:26.727535 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c4f1c-2def-4ab8-ae99-8e0ea3a32e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:26.761424 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-cb5c4f1c-2def-4ab8-ae99-8e0ea3a32e23 None None] MFA Rules not processed for user `453297dac64046d19fc1c331d2d85abc`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:18:26.763014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-cb5c4f1c-2def-4ab8-ae99-8e0ea3a32e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:18:26.773928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1223/2459] 10.222.0.44 () {58 vars in 1018 bytes} [Sun Sep 11 15:18:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:18:26.786746 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b003d5d-d99a-4ac6-b958-f895220ff7c9 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:26.788550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-4b003d5d-d99a-4ac6-b958-f895220ff7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:18:26.854220 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b003d5d-d99a-4ac6-b958-f895220ff7c9 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:26.858585 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041b7f1-9e54-4776-874d-88829ceaff82 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fe5293447346eaafd16818e5bd12f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:26.859505 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041b7f1-9e54-4776-874d-88829ceaff82 service glance] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:26.859844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041b7f1-9e54-4776-874d-88829ceaff82 service glance] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:26.860291 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041b7f1-9e54-4776-874d-88829ceaff82 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:26.874451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c041b7f1-9e54-4776-874d-88829ceaff82 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:18:26.875987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c041b7f1-9e54-4776-874d-88829ceaff82 service glance] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:18:26.887722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1237/2460] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:18:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:18:26.972269 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e3672-b42a-40ac-8c34-7e4ddb37cd37 None None] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:26.972850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e3672-b42a-40ac-8c34-7e4ddb37cd37 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:26.973326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e3672-b42a-40ac-8c34-7e4ddb37cd37 None None] PATH_INFO: `/` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:26.974885 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1224/2461] 10.222.0.44 () {54 vars in 921 bytes} [Sun Sep 11 15:18:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:18:26.980935 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57ed8a-6273-465e-bff3-107c49161191 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:26.981282 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57ed8a-6273-465e-bff3-107c49161191 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:26.981591 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57ed8a-6273-465e-bff3-107c49161191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:27.004320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-cf57ed8a-6273-465e-bff3-107c49161191 None None] MFA Rules not processed for user `697b73f182e6419a84e0c8c6b0fc0c06`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:18:27.005698 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-cf57ed8a-6273-465e-bff3-107c49161191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:18:27.013123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1238/2462] 10.222.0.44 () {58 vars in 1018 bytes} [Sun Sep 11 15:18:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:18:27.028542 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e3672-b42a-40ac-8c34-7e4ddb37cd37 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:27.031634 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-d46e3672-b42a-40ac-8c34-7e4ddb37cd37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:18:27.048679 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e3672-b42a-40ac-8c34-7e4ddb37cd37 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:27.053080 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f90200f-a953-4b5c-86df-82d2e5a77df9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:27.053747 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90200f-a953-4b5c-86df-82d2e5a77df9 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:27.054018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90200f-a953-4b5c-86df-82d2e5a77df9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:27.054281 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90200f-a953-4b5c-86df-82d2e5a77df9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:27.062131 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f90200f-a953-4b5c-86df-82d2e5a77df9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:18:27.063598 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f90200f-a953-4b5c-86df-82d2e5a77df9 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:18:27.071988 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1225/2463] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:18:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:18:29.370631 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8411a-d99c-4a24-acfc-b1f3824c158c None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:29.371127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8411a-d99c-4a24-acfc-b1f3824c158c None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:29.371218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8411a-d99c-4a24-acfc-b1f3824c158c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:29.391822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-8ba8411a-d99c-4a24-acfc-b1f3824c158c None None] MFA Rules not processed for user `3fada78ceaeb4f3bad77c516f6dfcfd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:18:29.393744 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-8ba8411a-d99c-4a24-acfc-b1f3824c158c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:18:29.403008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1239/2464] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 15:18:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:18:29.418641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f90200f-a953-4b5c-86df-82d2e5a77df9 service neutron] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:29.432511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f90200f-a953-4b5c-86df-82d2e5a77df9 service neutron] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:29.438832 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33154d-dc93-4402-8b5d-d287e290db86 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05aa5e40ee4542719657bc03b2015e72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:29.439441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33154d-dc93-4402-8b5d-d287e290db86 service swift] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:29.439717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33154d-dc93-4402-8b5d-d287e290db86 service swift] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:29.439990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33154d-dc93-4402-8b5d-d287e290db86 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:29.449958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b33154d-dc93-4402-8b5d-d287e290db86 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:18:29.451578 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b33154d-dc93-4402-8b5d-d287e290db86 service swift] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:18:29.459931 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1226/2465] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 15:18:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 15:18:39.385611 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=32 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:18:39.456686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=15 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:18:44.383716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba6106-a943-41f4-b2ef-bb73ae6d668e None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:44.384321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba6106-a943-41f4-b2ef-bb73ae6d668e None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:44.384321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba6106-a943-41f4-b2ef-bb73ae6d668e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:44.406770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-aaba6106-a943-41f4-b2ef-bb73ae6d668e None None] MFA Rules not processed for user `3fada78ceaeb4f3bad77c516f6dfcfd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:18:44.408045 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-aaba6106-a943-41f4-b2ef-bb73ae6d668e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:18:44.416173 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1240/2466] 10.222.0.44 () {58 vars in 967 bytes} [Sun Sep 11 15:18:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:18:44.430468 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33154d-dc93-4402-8b5d-d287e290db86 service swift] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:44.442519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33154d-dc93-4402-8b5d-d287e290db86 service swift] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:44.448714 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5c7ac-db42-49b4-b100-82241de49178 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05aa5e40ee4542719657bc03b2015e72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:44.449469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5c7ac-db42-49b4-b100-82241de49178 service swift] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:44.449746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5c7ac-db42-49b4-b100-82241de49178 service swift] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:44.450022 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5c7ac-db42-49b4-b100-82241de49178 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:44.459301 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5c7ac-db42-49b4-b100-82241de49178 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:18:44.460980 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5c7ac-db42-49b4-b100-82241de49178 service swift] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:18:44.470713 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1227/2467] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 15:18:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 15:18:50.748805 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba6106-a943-41f4-b2ef-bb73ae6d668e None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:50.760432 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba6106-a943-41f4-b2ef-bb73ae6d668e None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:50.766533 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d37c2e-61a3-459a-8406-e765738a0070 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:50.767380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d37c2e-61a3-459a-8406-e765738a0070 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:50.767656 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d37c2e-61a3-459a-8406-e765738a0070 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:50.767921 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d37c2e-61a3-459a-8406-e765738a0070 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:50.770949 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-09d37c2e-61a3-459a-8406-e765738a0070 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:18:50.785416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d37c2e-61a3-459a-8406-e765738a0070 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:18:50.786885 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d37c2e-61a3-459a-8406-e765738a0070 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:18:50.796802 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1241/2468] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:18:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:18:54.202495 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5c7ac-db42-49b4-b100-82241de49178 service swift] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:54.204011 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-e2e5c7ac-db42-49b4-b100-82241de49178 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:18:54.221434 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5c7ac-db42-49b4-b100-82241de49178 service swift] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:54.229591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88f005-3459-43c8-97b8-16fd2599dde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:54.229591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88f005-3459-43c8-97b8-16fd2599dde3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:54.229591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88f005-3459-43c8-97b8-16fd2599dde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:54.229591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88f005-3459-43c8-97b8-16fd2599dde3 admin admin] PATH_INFO: `/v3/users/c44105f2d90e469fb03f311a4cf1476f` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:54.231542 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f88f005-3459-43c8-97b8-16fd2599dde3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c44105f2d90e469fb03f311a4cf1476f)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:18:54.233619 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f88f005-3459-43c8-97b8-16fd2599dde3 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:18:54.296989 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3f88f005-3459-43c8-97b8-16fd2599dde3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c44105f2d90e469fb03f311a4cf1476f'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:18:54.302719 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3f88f005-3459-43c8-97b8-16fd2599dde3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c44105f2d90e469fb03f311a4cf1476f'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:18:54.315521 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3f88f005-3459-43c8-97b8-16fd2599dde3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c44105f2d90e469fb03f311a4cf1476f'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:18:54.316240 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3f88f005-3459-43c8-97b8-16fd2599dde3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c44105f2d90e469fb03f311a4cf1476f'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:18:54.323833 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3f88f005-3459-43c8-97b8-16fd2599dde3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c44105f2d90e469fb03f311a4cf1476f'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:18:54.330989 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1228/2469] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:18:54 2022] DELETE /identity/v3/users/c44105f2d90e469fb03f311a4cf1476f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:18:54.533465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d37c2e-61a3-459a-8406-e765738a0070 service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:54.545655 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d37c2e-61a3-459a-8406-e765738a0070 service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:54.605702 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7e23a-5c2d-4c1e-b7ea-813292dc4594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:54.606363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7e23a-5c2d-4c1e-b7ea-813292dc4594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:54.606633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7e23a-5c2d-4c1e-b7ea-813292dc4594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:54.606897 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7e23a-5c2d-4c1e-b7ea-813292dc4594 admin admin] PATH_INFO: `/v3/projects/c0693786c6694f9e8a8bb8bec8fba0b5` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:54.608008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7e23a-5c2d-4c1e-b7ea-813292dc4594 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0693786c6694f9e8a8bb8bec8fba0b5)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:18:54.609958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7e23a-5c2d-4c1e-b7ea-813292dc4594 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:18:54.657811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-54e7e23a-5c2d-4c1e-b7ea-813292dc4594 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0693786c6694f9e8a8bb8bec8fba0b5', 'request_id': 'req-54e7e23a-5c2d-4c1e-b7ea-813292dc4594'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:18:54.663632 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-54e7e23a-5c2d-4c1e-b7ea-813292dc4594 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0693786c6694f9e8a8bb8bec8fba0b5', 'request_id': 'req-54e7e23a-5c2d-4c1e-b7ea-813292dc4594'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:18:54.670562 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-54e7e23a-5c2d-4c1e-b7ea-813292dc4594 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0693786c6694f9e8a8bb8bec8fba0b5', 'request_id': 'req-54e7e23a-5c2d-4c1e-b7ea-813292dc4594'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:18:54.686611 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-54e7e23a-5c2d-4c1e-b7ea-813292dc4594 admin admin] The token cache is being invalidate because project c0693786c6694f9e8a8bb8bec8fba0b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:18:54.686899 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-54e7e23a-5c2d-4c1e-b7ea-813292dc4594 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:18:54.687549 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-54e7e23a-5c2d-4c1e-b7ea-813292dc4594 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:18:54.689074 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1242/2470] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:18:54 2022] DELETE /identity/v3/projects/c0693786c6694f9e8a8bb8bec8fba0b5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:18:54.703202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcf2f9-5720-4562-b4e0-3bf48aa6135a None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:54.703522 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcf2f9-5720-4562-b4e0-3bf48aa6135a None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:54.703850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcf2f9-5720-4562-b4e0-3bf48aa6135a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:54.729225 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-c6fcf2f9-5720-4562-b4e0-3bf48aa6135a None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:18:54.730454 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c6fcf2f9-5720-4562-b4e0-3bf48aa6135a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:18:54.801924 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1229/2471] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:18:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:18:54.810527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a8d34-1825-4187-982a-16c228e0c308 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:54.810812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a8d34-1825-4187-982a-16c228e0c308 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:54.811046 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a8d34-1825-4187-982a-16c228e0c308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:54.833462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-3b3a8d34-1825-4187-982a-16c228e0c308 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:18:54.834696 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-3b3a8d34-1825-4187-982a-16c228e0c308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:18:54.843140 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1243/2472] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:18:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:18:54.847657 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcf2f9-5720-4562-b4e0-3bf48aa6135a None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:54.857941 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcf2f9-5720-4562-b4e0-3bf48aa6135a None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:54.863002 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3c663-40a5-47f5-93f0-9b8c4f460ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:54.863596 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3c663-40a5-47f5-93f0-9b8c4f460ead admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:54.863874 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3c663-40a5-47f5-93f0-9b8c4f460ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:54.864150 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3c663-40a5-47f5-93f0-9b8c4f460ead admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:54.864790 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3c663-40a5-47f5-93f0-9b8c4f460ead admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:18:54.865091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3c663-40a5-47f5-93f0-9b8c4f460ead admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:18:54.866733 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3c663-40a5-47f5-93f0-9b8c4f460ead admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:18:54.878578 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1230/2473] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:18:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:18:54.885225 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a8d34-1825-4187-982a-16c228e0c308 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:54.894743 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a8d34-1825-4187-982a-16c228e0c308 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:54.900974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4b20e-4758-46f6-83db-e3d787fd8810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:54.901599 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4b20e-4758-46f6-83db-e3d787fd8810 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:54.901869 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4b20e-4758-46f6-83db-e3d787fd8810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:54.902128 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4b20e-4758-46f6-83db-e3d787fd8810 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:54.902749 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce4b20e-4758-46f6-83db-e3d787fd8810 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:18:54.904645 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce4b20e-4758-46f6-83db-e3d787fd8810 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:18:54.936086 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1244/2474] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:18:54 2022] POST /identity/v3/projects => generated 393 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:18:54.942685 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3c663-40a5-47f5-93f0-9b8c4f460ead admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:54.954668 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3c663-40a5-47f5-93f0-9b8c4f460ead admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:55.010135 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:18:55.026857 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d2c54-9739-4710-b03b-de5fbd24e8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:55.027744 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d2c54-9739-4710-b03b-de5fbd24e8f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:55.027859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d2c54-9739-4710-b03b-de5fbd24e8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:55.028488 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d2c54-9739-4710-b03b-de5fbd24e8f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:55.029166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2d2c54-9739-4710-b03b-de5fbd24e8f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:18:55.031160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2d2c54-9739-4710-b03b-de5fbd24e8f9 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:18:55.059043 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1231/2475] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:18:54 2022] POST /identity/v3/users => generated 297 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:18:55.060258 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4b20e-4758-46f6-83db-e3d787fd8810 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:55.071527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4b20e-4758-46f6-83db-e3d787fd8810 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:55.076835 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a80b7-1cd5-4301-bc89-1e8976dfa9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:55.077494 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a80b7-1cd5-4301-bc89-1e8976dfa9a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:55.077771 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a80b7-1cd5-4301-bc89-1e8976dfa9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:55.078034 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a80b7-1cd5-4301-bc89-1e8976dfa9a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:55.078685 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a80b7-1cd5-4301-bc89-1e8976dfa9a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:18:55.080670 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a80b7-1cd5-4301-bc89-1e8976dfa9a8 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:18:55.091399 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1245/2476] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:18:55.096523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d2c54-9739-4710-b03b-de5fbd24e8f9 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:55.112500 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d2c54-9739-4710-b03b-de5fbd24e8f9 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:55.117620 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c2069-90be-4df4-a82c-dd692e666df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:55.118262 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c2069-90be-4df4-a82c-dd692e666df6 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:55.118632 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c2069-90be-4df4-a82c-dd692e666df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:55.118911 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c2069-90be-4df4-a82c-dd692e666df6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:55.119559 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7c2069-90be-4df4-a82c-dd692e666df6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:18:55.121170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7c2069-90be-4df4-a82c-dd692e666df6 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:18:55.132860 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1232/2477] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:18:55.137290 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a80b7-1cd5-4301-bc89-1e8976dfa9a8 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:55.148827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a80b7-1cd5-4301-bc89-1e8976dfa9a8 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:55.154803 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b46d3-67a6-4fb6-806b-c3d14f1a17ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:55.155527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b46d3-67a6-4fb6-806b-c3d14f1a17ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:55.155825 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b46d3-67a6-4fb6-806b-c3d14f1a17ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:55.156116 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b46d3-67a6-4fb6-806b-c3d14f1a17ca admin admin] PATH_INFO: `/v3/projects/a808a30d0bf743cab647280ba58520ac/users/c49e5f813da043d4a14eee4782f697df/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:55.172342 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b46d3-67a6-4fb6-806b-c3d14f1a17ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=a808a30d0bf743cab647280ba58520ac, user_id=c49e5f813da043d4a14eee4782f697df, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:18:55.175040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b46d3-67a6-4fb6-806b-c3d14f1a17ca admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:18:55.195328 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1246/2478] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:18:55 2022] PUT /identity/v3/projects/a808a30d0bf743cab647280ba58520ac/users/c49e5f813da043d4a14eee4782f697df/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:18:55.206977 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac02b45-23ab-4805-bb15-e490a582795b None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:55.207584 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac02b45-23ab-4805-bb15-e490a582795b None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:55.207584 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac02b45-23ab-4805-bb15-e490a582795b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:55.239586 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-cac02b45-23ab-4805-bb15-e490a582795b None None] MFA Rules not processed for user `c49e5f813da043d4a14eee4782f697df`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:18:55.240857 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-cac02b45-23ab-4805-bb15-e490a582795b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:18:55.312962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1233/2479] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:18:55 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:18:55.318393 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b46d3-67a6-4fb6-806b-c3d14f1a17ca admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:55.328356 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:18:55.387890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b46d3-67a6-4fb6-806b-c3d14f1a17ca admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:55.393030 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ad648-3a54-44f2-8f27-c6ec00c90201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:55.393681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ad648-3a54-44f2-8f27-c6ec00c90201 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:55.393959 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ad648-3a54-44f2-8f27-c6ec00c90201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:55.394228 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ad648-3a54-44f2-8f27-c6ec00c90201 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:55.394876 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5ad648-3a54-44f2-8f27-c6ec00c90201 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:18:55.396797 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5ad648-3a54-44f2-8f27-c6ec00c90201 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:18:55.429975 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1247/2480] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:18:55 2022] POST /identity/v3/projects => generated 393 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:18:55.431607 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac02b45-23ab-4805-bb15-e490a582795b None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:55.444402 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac02b45-23ab-4805-bb15-e490a582795b None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:55.508602 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483103ad-2031-42d3-925c-013f38223c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:55.509745 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483103ad-2031-42d3-925c-013f38223c31 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:55.510012 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483103ad-2031-42d3-925c-013f38223c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:55.510325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483103ad-2031-42d3-925c-013f38223c31 admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:55.510962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483103ad-2031-42d3-925c-013f38223c31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:18:55.512900 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483103ad-2031-42d3-925c-013f38223c31 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:18:55.536733 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1234/2481] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:18:55 2022] POST /identity/v3/users => generated 303 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:18:55.541605 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ad648-3a54-44f2-8f27-c6ec00c90201 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:55.559095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ad648-3a54-44f2-8f27-c6ec00c90201 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:55.566993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8853bb-1bcb-4d84-86a8-c4723b449c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:55.567911 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8853bb-1bcb-4d84-86a8-c4723b449c56 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:55.568348 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8853bb-1bcb-4d84-86a8-c4723b449c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:55.568865 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8853bb-1bcb-4d84-86a8-c4723b449c56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:55.570027 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8853bb-1bcb-4d84-86a8-c4723b449c56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:18:55.572282 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8853bb-1bcb-4d84-86a8-c4723b449c56 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:18:55.584159 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1248/2482] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:18:55.590242 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483103ad-2031-42d3-925c-013f38223c31 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:55.601563 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483103ad-2031-42d3-925c-013f38223c31 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:55.607404 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562fefe5-6984-47ac-97e3-3e578d6d5f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:55.608258 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fefe5-6984-47ac-97e3-3e578d6d5f70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:55.608845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fefe5-6984-47ac-97e3-3e578d6d5f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:55.609240 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fefe5-6984-47ac-97e3-3e578d6d5f70 admin admin] PATH_INFO: `/v3/projects/07051fbe39024eb98fec61cd7b8d7516/users/9228d28b44b6419b83e04617881300e6/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:55.627629 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562fefe5-6984-47ac-97e3-3e578d6d5f70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07051fbe39024eb98fec61cd7b8d7516, user_id=9228d28b44b6419b83e04617881300e6, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:18:55.631038 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562fefe5-6984-47ac-97e3-3e578d6d5f70 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:18:55.645337 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1235/2483] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:18:55 2022] PUT /identity/v3/projects/07051fbe39024eb98fec61cd7b8d7516/users/9228d28b44b6419b83e04617881300e6/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:18:55.649495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8853bb-1bcb-4d84-86a8-c4723b449c56 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:55.667772 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8853bb-1bcb-4d84-86a8-c4723b449c56 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:55.722503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c78aaf-5cc5-44c0-92f4-af25120bf228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:55.723192 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c78aaf-5cc5-44c0-92f4-af25120bf228 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:55.723453 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c78aaf-5cc5-44c0-92f4-af25120bf228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:55.723792 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c78aaf-5cc5-44c0-92f4-af25120bf228 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:55.724779 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c78aaf-5cc5-44c0-92f4-af25120bf228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:18:55.727609 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c78aaf-5cc5-44c0-92f4-af25120bf228 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:18:55.742533 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1249/2484] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:18:55.747551 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562fefe5-6984-47ac-97e3-3e578d6d5f70 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:55.758690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562fefe5-6984-47ac-97e3-3e578d6d5f70 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:55.763472 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675ff38-ecca-452c-8dd2-f6fdc772d0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:55.764147 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675ff38-ecca-452c-8dd2-f6fdc772d0f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:55.764421 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675ff38-ecca-452c-8dd2-f6fdc772d0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:55.764688 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675ff38-ecca-452c-8dd2-f6fdc772d0f8 admin admin] PATH_INFO: `/v3/projects/07051fbe39024eb98fec61cd7b8d7516/users/9228d28b44b6419b83e04617881300e6/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:55.766680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4675ff38-ecca-452c-8dd2-f6fdc772d0f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07051fbe39024eb98fec61cd7b8d7516, user_id=9228d28b44b6419b83e04617881300e6, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:18:55.769344 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4675ff38-ecca-452c-8dd2-f6fdc772d0f8 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:18:55.783250 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1236/2485] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:18:55 2022] PUT /identity/v3/projects/07051fbe39024eb98fec61cd7b8d7516/users/9228d28b44b6419b83e04617881300e6/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:18:55.795337 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592c8e9-763d-4a7c-abe1-196ebc13bc29 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:55.795616 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592c8e9-763d-4a7c-abe1-196ebc13bc29 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:55.795888 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592c8e9-763d-4a7c-abe1-196ebc13bc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:55.828633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-0592c8e9-763d-4a7c-abe1-196ebc13bc29 None None] MFA Rules not processed for user `9228d28b44b6419b83e04617881300e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:18:55.829870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-0592c8e9-763d-4a7c-abe1-196ebc13bc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:18:55.901838 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1250/2486] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:18:55 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:18:55.914080 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675ff38-ecca-452c-8dd2-f6fdc772d0f8 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:55.917894 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-4675ff38-ecca-452c-8dd2-f6fdc772d0f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:18:55.929042 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:18:55.985253 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675ff38-ecca-452c-8dd2-f6fdc772d0f8 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:55.989243 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3544c25-be8c-433e-b706-f41ef25b0e86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:55.989889 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3544c25-be8c-433e-b706-f41ef25b0e86 service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:55.990167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3544c25-be8c-433e-b706-f41ef25b0e86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:55.990427 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3544c25-be8c-433e-b706-f41ef25b0e86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:56.001511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3544c25-be8c-433e-b706-f41ef25b0e86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:18:56.002960 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3544c25-be8c-433e-b706-f41ef25b0e86 service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:18:56.012948 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1237/2487] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:18:55 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:18:56.060875 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0592c8e9-763d-4a7c-abe1-196ebc13bc29 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:56.062415 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-0592c8e9-763d-4a7c-abe1-196ebc13bc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:18:56.070282 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:18:56.076965 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0592c8e9-763d-4a7c-abe1-196ebc13bc29 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:56.081160 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398483c-bcaf-4326-b9be-e2ce91676e5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:56.081794 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398483c-bcaf-4326-b9be-e2ce91676e5e service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:56.082067 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398483c-bcaf-4326-b9be-e2ce91676e5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:56.082340 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398483c-bcaf-4326-b9be-e2ce91676e5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:56.147946 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7398483c-bcaf-4326-b9be-e2ce91676e5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:18:56.150311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7398483c-bcaf-4326-b9be-e2ce91676e5e service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:18:56.162739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1251/2488] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:18:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:18:59.656135 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3544c25-be8c-433e-b706-f41ef25b0e86 service cinder] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:59.724026 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3544c25-be8c-433e-b706-f41ef25b0e86 service cinder] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:59.728817 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9e93a-53b8-463f-a761-a5820475be83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:59.729467 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9e93a-53b8-463f-a761-a5820475be83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:59.729739 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9e93a-53b8-463f-a761-a5820475be83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:59.730001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9e93a-53b8-463f-a761-a5820475be83 admin admin] PATH_INFO: `/v3/users/c49e5f813da043d4a14eee4782f697df` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:59.731140 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9e93a-53b8-463f-a761-a5820475be83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c49e5f813da043d4a14eee4782f697df)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:18:59.733061 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9e93a-53b8-463f-a761-a5820475be83 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:18:59.780641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-2cc9e93a-53b8-463f-a761-a5820475be83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c49e5f813da043d4a14eee4782f697df'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:18:59.788563 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-2cc9e93a-53b8-463f-a761-a5820475be83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c49e5f813da043d4a14eee4782f697df'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:18:59.801551 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-2cc9e93a-53b8-463f-a761-a5820475be83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c49e5f813da043d4a14eee4782f697df'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:18:59.802331 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-2cc9e93a-53b8-463f-a761-a5820475be83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c49e5f813da043d4a14eee4782f697df'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:18:59.809527 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-2cc9e93a-53b8-463f-a761-a5820475be83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c49e5f813da043d4a14eee4782f697df'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:18:59.816076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1238/2489] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:18:59 2022] DELETE /identity/v3/users/c49e5f813da043d4a14eee4782f697df => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:18:59.820679 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58767578-ecab-4944-b345-b2a5a548f923 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:59.821203 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58767578-ecab-4944-b345-b2a5a548f923 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:59.821311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58767578-ecab-4944-b345-b2a5a548f923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:18:59.840608 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-58767578-ecab-4944-b345-b2a5a548f923 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:18:59.841855 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-58767578-ecab-4944-b345-b2a5a548f923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:18:59.909984 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1252/2490] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:18:59.923373 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9e93a-53b8-463f-a761-a5820475be83 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:18:59.924876 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-2cc9e93a-53b8-463f-a761-a5820475be83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:18:59.958977 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:18:59.992555 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9e93a-53b8-463f-a761-a5820475be83 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:18:59.993616 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e107cfd-5753-4022-a655-a0ac5008e6b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:18:59.994280 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e107cfd-5753-4022-a655-a0ac5008e6b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:18:59.994550 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e107cfd-5753-4022-a655-a0ac5008e6b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:18:59.994817 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e107cfd-5753-4022-a655-a0ac5008e6b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:00.005294 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e107cfd-5753-4022-a655-a0ac5008e6b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:00.006725 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e107cfd-5753-4022-a655-a0ac5008e6b5 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:00.016787 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1239/2491] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:18:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:19:00.313724 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58767578-ecab-4944-b345-b2a5a548f923 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:00.315775 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-58767578-ecab-4944-b345-b2a5a548f923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:00.328870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58767578-ecab-4944-b345-b2a5a548f923 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:00.345635 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0118e75f-9d12-4157-acb6-8f1de278014e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:00.346307 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118e75f-9d12-4157-acb6-8f1de278014e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:00.346584 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118e75f-9d12-4157-acb6-8f1de278014e admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:00.346872 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118e75f-9d12-4157-acb6-8f1de278014e admin admin] PATH_INFO: `/v3/projects/a808a30d0bf743cab647280ba58520ac` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:00.348112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0118e75f-9d12-4157-acb6-8f1de278014e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a808a30d0bf743cab647280ba58520ac)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:00.350119 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0118e75f-9d12-4157-acb6-8f1de278014e admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:00.397825 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-0118e75f-9d12-4157-acb6-8f1de278014e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a808a30d0bf743cab647280ba58520ac', 'request_id': 'req-0118e75f-9d12-4157-acb6-8f1de278014e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:00.403996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-0118e75f-9d12-4157-acb6-8f1de278014e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a808a30d0bf743cab647280ba58520ac', 'request_id': 'req-0118e75f-9d12-4157-acb6-8f1de278014e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:00.410652 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-0118e75f-9d12-4157-acb6-8f1de278014e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a808a30d0bf743cab647280ba58520ac', 'request_id': 'req-0118e75f-9d12-4157-acb6-8f1de278014e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:00.425954 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-0118e75f-9d12-4157-acb6-8f1de278014e admin admin] The token cache is being invalidate because project a808a30d0bf743cab647280ba58520ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:19:00.426152 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-0118e75f-9d12-4157-acb6-8f1de278014e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:00.426773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-0118e75f-9d12-4157-acb6-8f1de278014e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:00.428276 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1253/2492] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:19:00 2022] DELETE /identity/v3/projects/a808a30d0bf743cab647280ba58520ac => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:00.434611 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e107cfd-5753-4022-a655-a0ac5008e6b5 service neutron] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:00.445706 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e107cfd-5753-4022-a655-a0ac5008e6b5 service neutron] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:00.503525 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08121a6c-4885-400f-bfed-53efce35820f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:00.504157 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08121a6c-4885-400f-bfed-53efce35820f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:00.504433 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08121a6c-4885-400f-bfed-53efce35820f admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:00.504697 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08121a6c-4885-400f-bfed-53efce35820f admin admin] PATH_INFO: `/v3/users/9228d28b44b6419b83e04617881300e6` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:00.505873 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08121a6c-4885-400f-bfed-53efce35820f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9228d28b44b6419b83e04617881300e6)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:00.507795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08121a6c-4885-400f-bfed-53efce35820f admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:00.556590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-08121a6c-4885-400f-bfed-53efce35820f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9228d28b44b6419b83e04617881300e6'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:00.565077 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-08121a6c-4885-400f-bfed-53efce35820f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9228d28b44b6419b83e04617881300e6'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:00.577858 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-08121a6c-4885-400f-bfed-53efce35820f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9228d28b44b6419b83e04617881300e6'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:00.578654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-08121a6c-4885-400f-bfed-53efce35820f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9228d28b44b6419b83e04617881300e6'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:00.586854 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-08121a6c-4885-400f-bfed-53efce35820f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9228d28b44b6419b83e04617881300e6'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:00.598235 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1240/2493] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:19:00 2022] DELETE /identity/v3/users/9228d28b44b6419b83e04617881300e6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:00.904796 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0118e75f-9d12-4157-acb6-8f1de278014e admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:00.906360 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-0118e75f-9d12-4157-acb6-8f1de278014e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:00.919591 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0118e75f-9d12-4157-acb6-8f1de278014e admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:00.941745 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:00.980262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a969d-022c-4f05-b1b8-f480e1a61a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:00.980820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a969d-022c-4f05-b1b8-f480e1a61a73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:00.981091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a969d-022c-4f05-b1b8-f480e1a61a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:00.981396 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a969d-022c-4f05-b1b8-f480e1a61a73 admin admin] PATH_INFO: `/v3/projects/07051fbe39024eb98fec61cd7b8d7516` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:00.982510 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a969d-022c-4f05-b1b8-f480e1a61a73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07051fbe39024eb98fec61cd7b8d7516)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:00.984419 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a969d-022c-4f05-b1b8-f480e1a61a73 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:01.032895 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-859a969d-022c-4f05-b1b8-f480e1a61a73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07051fbe39024eb98fec61cd7b8d7516', 'request_id': 'req-859a969d-022c-4f05-b1b8-f480e1a61a73'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:01.038973 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-859a969d-022c-4f05-b1b8-f480e1a61a73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07051fbe39024eb98fec61cd7b8d7516', 'request_id': 'req-859a969d-022c-4f05-b1b8-f480e1a61a73'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:01.047503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-859a969d-022c-4f05-b1b8-f480e1a61a73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07051fbe39024eb98fec61cd7b8d7516', 'request_id': 'req-859a969d-022c-4f05-b1b8-f480e1a61a73'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:01.063207 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-859a969d-022c-4f05-b1b8-f480e1a61a73 admin admin] The token cache is being invalidate because project 07051fbe39024eb98fec61cd7b8d7516 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:19:01.063488 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-859a969d-022c-4f05-b1b8-f480e1a61a73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:01.064093 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-859a969d-022c-4f05-b1b8-f480e1a61a73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:01.066059 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1254/2494] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:19:00 2022] DELETE /identity/v3/projects/07051fbe39024eb98fec61cd7b8d7516 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:01.071847 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce82e3f-fceb-4c31-9108-7ccb1eee4784 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:01.072389 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce82e3f-fceb-4c31-9108-7ccb1eee4784 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:01.072489 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce82e3f-fceb-4c31-9108-7ccb1eee4784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:01.093268 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-2ce82e3f-fceb-4c31-9108-7ccb1eee4784 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:01.094485 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-2ce82e3f-fceb-4c31-9108-7ccb1eee4784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:01.147021 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:01.165660 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1241/2495] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:01.173675 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d11ae-c910-4ce3-88f7-2fe2f3806305 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:01.173927 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d11ae-c910-4ce3-88f7-2fe2f3806305 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:01.174197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d11ae-c910-4ce3-88f7-2fe2f3806305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:01.195556 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-538d11ae-c910-4ce3-88f7-2fe2f3806305 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:01.196802 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-538d11ae-c910-4ce3-88f7-2fe2f3806305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:01.206211 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1255/2496] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:01.210968 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce82e3f-fceb-4c31-9108-7ccb1eee4784 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:01.224100 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce82e3f-fceb-4c31-9108-7ccb1eee4784 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:01.229690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf53e1b-e589-4bb8-b5cf-b55cf94ed56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:01.230464 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf53e1b-e589-4bb8-b5cf-b55cf94ed56d admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:01.230607 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf53e1b-e589-4bb8-b5cf-b55cf94ed56d admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:01.230859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf53e1b-e589-4bb8-b5cf-b55cf94ed56d admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:01.231507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf53e1b-e589-4bb8-b5cf-b55cf94ed56d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:19:01.231807 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf53e1b-e589-4bb8-b5cf-b55cf94ed56d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:01.233505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf53e1b-e589-4bb8-b5cf-b55cf94ed56d admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:01.248216 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1242/2497] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:19:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:19:01.253105 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d11ae-c910-4ce3-88f7-2fe2f3806305 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:01.263201 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d11ae-c910-4ce3-88f7-2fe2f3806305 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:01.268490 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa50f76-6377-4f01-82fb-f5683bea80b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:01.269084 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa50f76-6377-4f01-82fb-f5683bea80b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:01.269406 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa50f76-6377-4f01-82fb-f5683bea80b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:01.269676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa50f76-6377-4f01-82fb-f5683bea80b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:01.270319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa50f76-6377-4f01-82fb-f5683bea80b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:01.272217 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa50f76-6377-4f01-82fb-f5683bea80b1 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:01.306933 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1256/2498] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:19:01 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:19:01.311520 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf53e1b-e589-4bb8-b5cf-b55cf94ed56d admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:01.322171 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf53e1b-e589-4bb8-b5cf-b55cf94ed56d admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:01.378817 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96634124-2f29-4702-afb1-5fe6d50151e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:01.379436 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96634124-2f29-4702-afb1-5fe6d50151e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:01.379710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96634124-2f29-4702-afb1-5fe6d50151e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:01.379975 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96634124-2f29-4702-afb1-5fe6d50151e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:01.380611 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96634124-2f29-4702-afb1-5fe6d50151e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:01.382587 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96634124-2f29-4702-afb1-5fe6d50151e9 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:01.408818 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1243/2499] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:19:01 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:19:01.410666 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa50f76-6377-4f01-82fb-f5683bea80b1 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:01.422166 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa50f76-6377-4f01-82fb-f5683bea80b1 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:01.427320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b8a02a-68da-4dcf-a986-bd5989211c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:01.427943 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8a02a-68da-4dcf-a986-bd5989211c29 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:01.428214 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8a02a-68da-4dcf-a986-bd5989211c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:01.428476 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8a02a-68da-4dcf-a986-bd5989211c29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:01.429116 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b8a02a-68da-4dcf-a986-bd5989211c29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:01.430744 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b8a02a-68da-4dcf-a986-bd5989211c29 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:01.444730 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1257/2500] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:19:01.450673 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96634124-2f29-4702-afb1-5fe6d50151e9 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:01.461022 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96634124-2f29-4702-afb1-5fe6d50151e9 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:01.466166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac708cb8-6b87-4f8f-928d-4789497c6764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:01.466767 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac708cb8-6b87-4f8f-928d-4789497c6764 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:01.467039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac708cb8-6b87-4f8f-928d-4789497c6764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:01.467322 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac708cb8-6b87-4f8f-928d-4789497c6764 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:01.467930 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac708cb8-6b87-4f8f-928d-4789497c6764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:01.469602 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac708cb8-6b87-4f8f-928d-4789497c6764 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:01.482342 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1244/2501] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:19:01.486900 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b8a02a-68da-4dcf-a986-bd5989211c29 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:01.497472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b8a02a-68da-4dcf-a986-bd5989211c29 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:01.502636 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e98039-8504-44aa-885a-b0dd30d45d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:01.503304 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e98039-8504-44aa-885a-b0dd30d45d64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:01.503576 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e98039-8504-44aa-885a-b0dd30d45d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:01.503845 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e98039-8504-44aa-885a-b0dd30d45d64 admin admin] PATH_INFO: `/v3/projects/209ba3c283454a0c8235da553e0a0090/users/363d31b3cc444c9d9887cd80fda8b84f/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:01.519935 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e98039-8504-44aa-885a-b0dd30d45d64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=209ba3c283454a0c8235da553e0a0090, user_id=363d31b3cc444c9d9887cd80fda8b84f, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:01.522655 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e98039-8504-44aa-885a-b0dd30d45d64 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:01.540719 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1258/2502] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:19:01 2022] PUT /identity/v3/projects/209ba3c283454a0c8235da553e0a0090/users/363d31b3cc444c9d9887cd80fda8b84f/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:19:01.554620 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f05779-fe97-4d9b-ae4d-6401e78b98a8 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:01.555116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f05779-fe97-4d9b-ae4d-6401e78b98a8 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:01.555614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f05779-fe97-4d9b-ae4d-6401e78b98a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:01.591175 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-f6f05779-fe97-4d9b-ae4d-6401e78b98a8 None None] MFA Rules not processed for user `363d31b3cc444c9d9887cd80fda8b84f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:01.592431 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f6f05779-fe97-4d9b-ae4d-6401e78b98a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:01.664982 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1245/2503] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:01.670527 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e98039-8504-44aa-885a-b0dd30d45d64 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:01.726149 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:01.740549 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e98039-8504-44aa-885a-b0dd30d45d64 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:01.745995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019830fd-4947-486f-baf6-d23076afd65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:01.746614 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019830fd-4947-486f-baf6-d23076afd65a admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:01.747051 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019830fd-4947-486f-baf6-d23076afd65a admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:01.747323 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019830fd-4947-486f-baf6-d23076afd65a admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:01.747960 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019830fd-4947-486f-baf6-d23076afd65a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:01.749999 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019830fd-4947-486f-baf6-d23076afd65a admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:01.779703 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1259/2504] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:19:01 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:19:01.784286 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f05779-fe97-4d9b-ae4d-6401e78b98a8 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:01.795717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f05779-fe97-4d9b-ae4d-6401e78b98a8 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:01.844434 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:01.860389 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d815a1-97ea-4c7a-9eb2-089aab360a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:01.861004 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d815a1-97ea-4c7a-9eb2-089aab360a3c admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:01.861308 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d815a1-97ea-4c7a-9eb2-089aab360a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:01.861609 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d815a1-97ea-4c7a-9eb2-089aab360a3c admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:01.862206 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d815a1-97ea-4c7a-9eb2-089aab360a3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:01.864150 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d815a1-97ea-4c7a-9eb2-089aab360a3c admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:01.887604 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1246/2505] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:19:01 2022] POST /identity/v3/users => generated 298 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:19:01.893354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019830fd-4947-486f-baf6-d23076afd65a admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:01.908491 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019830fd-4947-486f-baf6-d23076afd65a admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:01.914169 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2215ae0-e2e6-4937-ab77-7a480a7406b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:01.914899 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2215ae0-e2e6-4937-ab77-7a480a7406b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:01.915261 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2215ae0-e2e6-4937-ab77-7a480a7406b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:01.915616 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2215ae0-e2e6-4937-ab77-7a480a7406b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:01.916650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2215ae0-e2e6-4937-ab77-7a480a7406b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:01.918977 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2215ae0-e2e6-4937-ab77-7a480a7406b6 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:01.932113 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1260/2506] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:19:01.939078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d815a1-97ea-4c7a-9eb2-089aab360a3c admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:01.951551 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d815a1-97ea-4c7a-9eb2-089aab360a3c admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:01.958996 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b6de10-fc14-46b3-b7f9-8a65bf66f3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:01.959967 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6de10-fc14-46b3-b7f9-8a65bf66f3ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:01.960347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6de10-fc14-46b3-b7f9-8a65bf66f3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:01.960690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6de10-fc14-46b3-b7f9-8a65bf66f3ca admin admin] PATH_INFO: `/v3/projects/c209de842bf348dcbdc1b592630b51b5/users/c00eb98ceffe4a378f1520433b66b609/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:01.980695 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b6de10-fc14-46b3-b7f9-8a65bf66f3ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=c209de842bf348dcbdc1b592630b51b5, user_id=c00eb98ceffe4a378f1520433b66b609, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:01.984853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b6de10-fc14-46b3-b7f9-8a65bf66f3ca admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:02.001983 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1247/2507] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:19:01 2022] PUT /identity/v3/projects/c209de842bf348dcbdc1b592630b51b5/users/c00eb98ceffe4a378f1520433b66b609/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:19:02.006099 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2215ae0-e2e6-4937-ab77-7a480a7406b6 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:02.016528 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2215ae0-e2e6-4937-ab77-7a480a7406b6 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:02.085849 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd0cf01-9eff-4712-87c0-584fe79a90e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:02.086457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0cf01-9eff-4712-87c0-584fe79a90e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:02.086732 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0cf01-9eff-4712-87c0-584fe79a90e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:02.087006 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0cf01-9eff-4712-87c0-584fe79a90e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:02.087644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd0cf01-9eff-4712-87c0-584fe79a90e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:02.089317 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd0cf01-9eff-4712-87c0-584fe79a90e4 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:02.099521 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1261/2508] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:19:02.104641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b6de10-fc14-46b3-b7f9-8a65bf66f3ca admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:02.117339 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b6de10-fc14-46b3-b7f9-8a65bf66f3ca admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:02.123090 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c67ba-02ba-459c-a1f8-820cfe3f824f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:02.123739 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c67ba-02ba-459c-a1f8-820cfe3f824f admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:02.124188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c67ba-02ba-459c-a1f8-820cfe3f824f admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:02.124643 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c67ba-02ba-459c-a1f8-820cfe3f824f admin admin] PATH_INFO: `/v3/projects/c209de842bf348dcbdc1b592630b51b5/users/c00eb98ceffe4a378f1520433b66b609/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:02.126960 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c67ba-02ba-459c-a1f8-820cfe3f824f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c209de842bf348dcbdc1b592630b51b5, user_id=c00eb98ceffe4a378f1520433b66b609, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:02.130151 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c67ba-02ba-459c-a1f8-820cfe3f824f admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:02.142341 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1248/2509] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:19:02 2022] PUT /identity/v3/projects/c209de842bf348dcbdc1b592630b51b5/users/c00eb98ceffe4a378f1520433b66b609/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:02.153775 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f0a61-a229-4146-aa97-ca50ef4c1cd0 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:02.154053 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f0a61-a229-4146-aa97-ca50ef4c1cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:02.154322 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f0a61-a229-4146-aa97-ca50ef4c1cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:02.183953 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-6b4f0a61-a229-4146-aa97-ca50ef4c1cd0 None None] MFA Rules not processed for user `c00eb98ceffe4a378f1520433b66b609`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:02.185162 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-6b4f0a61-a229-4146-aa97-ca50ef4c1cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:02.268396 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1262/2510] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:02 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:19:02.280658 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c67ba-02ba-459c-a1f8-820cfe3f824f admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:02.282867 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b22c67ba-02ba-459c-a1f8-820cfe3f824f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:02.349112 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c67ba-02ba-459c-a1f8-820cfe3f824f admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:02.355878 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec23980a-42f5-4f9f-b2f7-c825f434c955 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:02.356496 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23980a-42f5-4f9f-b2f7-c825f434c955 service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:02.356847 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23980a-42f5-4f9f-b2f7-c825f434c955 service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:02.357529 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23980a-42f5-4f9f-b2f7-c825f434c955 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:02.368006 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec23980a-42f5-4f9f-b2f7-c825f434c955 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:02.369295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec23980a-42f5-4f9f-b2f7-c825f434c955 service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:02.379088 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1249/2511] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:19:02 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:19:02.487984 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f0a61-a229-4146-aa97-ca50ef4c1cd0 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:02.498118 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f0a61-a229-4146-aa97-ca50ef4c1cd0 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:02.502147 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb8123-05f9-41af-b473-2bfe5b692589 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:02.502739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb8123-05f9-41af-b473-2bfe5b692589 service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:02.503011 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb8123-05f9-41af-b473-2bfe5b692589 service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:02.503284 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb8123-05f9-41af-b473-2bfe5b692589 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:02.548367 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:02.563524 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb8123-05f9-41af-b473-2bfe5b692589 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:02.565319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb8123-05f9-41af-b473-2bfe5b692589 service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:02.574987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1263/2512] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:19:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:19:05.199373 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec23980a-42f5-4f9f-b2f7-c825f434c955 service cinder] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:05.267281 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec23980a-42f5-4f9f-b2f7-c825f434c955 service cinder] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:05.272123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb02a3-da85-4560-8f82-94c51b419303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:05.272752 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb02a3-da85-4560-8f82-94c51b419303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:05.273021 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb02a3-da85-4560-8f82-94c51b419303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:05.273312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb02a3-da85-4560-8f82-94c51b419303 admin admin] PATH_INFO: `/v3/users/363d31b3cc444c9d9887cd80fda8b84f` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:05.274445 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb02a3-da85-4560-8f82-94c51b419303 admin admin] RBAC: Authorizing `identity:delete_user(user_id=363d31b3cc444c9d9887cd80fda8b84f)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:05.276360 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb02a3-da85-4560-8f82-94c51b419303 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:05.323201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:05.327169 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-f1eb02a3-da85-4560-8f82-94c51b419303 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '363d31b3cc444c9d9887cd80fda8b84f'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:05.336852 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-f1eb02a3-da85-4560-8f82-94c51b419303 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '363d31b3cc444c9d9887cd80fda8b84f'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:05.349862 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-f1eb02a3-da85-4560-8f82-94c51b419303 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '363d31b3cc444c9d9887cd80fda8b84f'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:05.350617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-f1eb02a3-da85-4560-8f82-94c51b419303 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '363d31b3cc444c9d9887cd80fda8b84f'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:05.358034 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-f1eb02a3-da85-4560-8f82-94c51b419303 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '363d31b3cc444c9d9887cd80fda8b84f'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:05.364735 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1250/2513] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:19:05 2022] DELETE /identity/v3/users/363d31b3cc444c9d9887cd80fda8b84f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:05.369391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7e06a-8ffe-444f-8db6-dc246499ada4 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:05.369891 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7e06a-8ffe-444f-8db6-dc246499ada4 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:05.369986 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7e06a-8ffe-444f-8db6-dc246499ada4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:05.390532 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-1fe7e06a-8ffe-444f-8db6-dc246499ada4 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:05.391896 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-1fe7e06a-8ffe-444f-8db6-dc246499ada4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:05.454956 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1264/2514] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:05.464912 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb02a3-da85-4560-8f82-94c51b419303 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:05.466538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f1eb02a3-da85-4560-8f82-94c51b419303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:05.524240 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb02a3-da85-4560-8f82-94c51b419303 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:05.528164 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462915b-0e8a-469f-9f28-6d0f13035760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:05.528767 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462915b-0e8a-469f-9f28-6d0f13035760 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:05.529037 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462915b-0e8a-469f-9f28-6d0f13035760 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:05.529330 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462915b-0e8a-469f-9f28-6d0f13035760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:05.538980 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462915b-0e8a-469f-9f28-6d0f13035760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:05.540405 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462915b-0e8a-469f-9f28-6d0f13035760 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:05.550167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1251/2515] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:19:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:19:05.848012 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe7e06a-8ffe-444f-8db6-dc246499ada4 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:05.849742 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-1fe7e06a-8ffe-444f-8db6-dc246499ada4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:05.869758 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe7e06a-8ffe-444f-8db6-dc246499ada4 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:05.875971 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16365be7-ba62-4315-92a3-729babf94ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:05.876677 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16365be7-ba62-4315-92a3-729babf94ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:05.876946 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16365be7-ba62-4315-92a3-729babf94ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:05.877164 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16365be7-ba62-4315-92a3-729babf94ee3 admin admin] PATH_INFO: `/v3/projects/209ba3c283454a0c8235da553e0a0090` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:05.878301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16365be7-ba62-4315-92a3-729babf94ee3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=209ba3c283454a0c8235da553e0a0090)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:05.880660 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16365be7-ba62-4315-92a3-729babf94ee3 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:05.937471 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-16365be7-ba62-4315-92a3-729babf94ee3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '209ba3c283454a0c8235da553e0a0090', 'request_id': 'req-16365be7-ba62-4315-92a3-729babf94ee3'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:05.946726 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-16365be7-ba62-4315-92a3-729babf94ee3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '209ba3c283454a0c8235da553e0a0090', 'request_id': 'req-16365be7-ba62-4315-92a3-729babf94ee3'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:05.955471 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-16365be7-ba62-4315-92a3-729babf94ee3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '209ba3c283454a0c8235da553e0a0090', 'request_id': 'req-16365be7-ba62-4315-92a3-729babf94ee3'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:05.971125 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-16365be7-ba62-4315-92a3-729babf94ee3 admin admin] The token cache is being invalidate because project 209ba3c283454a0c8235da553e0a0090 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:19:05.971346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-16365be7-ba62-4315-92a3-729babf94ee3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:05.972074 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-16365be7-ba62-4315-92a3-729babf94ee3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:05.974320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1265/2516] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:19:05 2022] DELETE /identity/v3/projects/209ba3c283454a0c8235da553e0a0090 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:05.979127 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462915b-0e8a-469f-9f28-6d0f13035760 service neutron] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:05.990644 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462915b-0e8a-469f-9f28-6d0f13035760 service neutron] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:06.048496 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f7cf6-3d0b-47f8-a0a7-8a61913b1664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:06.049160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f7cf6-3d0b-47f8-a0a7-8a61913b1664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:06.049444 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f7cf6-3d0b-47f8-a0a7-8a61913b1664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:06.049674 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f7cf6-3d0b-47f8-a0a7-8a61913b1664 admin admin] PATH_INFO: `/v3/users/c00eb98ceffe4a378f1520433b66b609` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:06.050874 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f7cf6-3d0b-47f8-a0a7-8a61913b1664 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c00eb98ceffe4a378f1520433b66b609)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:06.052789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f7cf6-3d0b-47f8-a0a7-8a61913b1664 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:06.102922 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-312f7cf6-3d0b-47f8-a0a7-8a61913b1664 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c00eb98ceffe4a378f1520433b66b609'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:06.111649 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-312f7cf6-3d0b-47f8-a0a7-8a61913b1664 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c00eb98ceffe4a378f1520433b66b609'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:06.119385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:06.124582 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-312f7cf6-3d0b-47f8-a0a7-8a61913b1664 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c00eb98ceffe4a378f1520433b66b609'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:06.125311 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-312f7cf6-3d0b-47f8-a0a7-8a61913b1664 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c00eb98ceffe4a378f1520433b66b609'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:06.133914 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-312f7cf6-3d0b-47f8-a0a7-8a61913b1664 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c00eb98ceffe4a378f1520433b66b609'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:06.140146 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1252/2517] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:19:05 2022] DELETE /identity/v3/users/c00eb98ceffe4a378f1520433b66b609 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:06.420427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16365be7-ba62-4315-92a3-729babf94ee3 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:06.421988 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-16365be7-ba62-4315-92a3-729babf94ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:06.437072 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16365be7-ba62-4315-92a3-729babf94ee3 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:06.500275 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5b250-3e35-4785-89c1-ef91fed34319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:06.500978 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5b250-3e35-4785-89c1-ef91fed34319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:06.501291 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5b250-3e35-4785-89c1-ef91fed34319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:06.502820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5b250-3e35-4785-89c1-ef91fed34319 admin admin] PATH_INFO: `/v3/projects/c209de842bf348dcbdc1b592630b51b5` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:06.506066 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c5b250-3e35-4785-89c1-ef91fed34319 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c209de842bf348dcbdc1b592630b51b5)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:06.508019 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c5b250-3e35-4785-89c1-ef91fed34319 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:06.523414 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:06.571135 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-03c5b250-3e35-4785-89c1-ef91fed34319 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c209de842bf348dcbdc1b592630b51b5', 'request_id': 'req-03c5b250-3e35-4785-89c1-ef91fed34319'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:06.580242 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-03c5b250-3e35-4785-89c1-ef91fed34319 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c209de842bf348dcbdc1b592630b51b5', 'request_id': 'req-03c5b250-3e35-4785-89c1-ef91fed34319'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:06.587174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-03c5b250-3e35-4785-89c1-ef91fed34319 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c209de842bf348dcbdc1b592630b51b5', 'request_id': 'req-03c5b250-3e35-4785-89c1-ef91fed34319'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:06.601990 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-03c5b250-3e35-4785-89c1-ef91fed34319 admin admin] The token cache is being invalidate because project c209de842bf348dcbdc1b592630b51b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:19:06.602278 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-03c5b250-3e35-4785-89c1-ef91fed34319 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:06.602915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-03c5b250-3e35-4785-89c1-ef91fed34319 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:06.604447 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1266/2518] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:19:06 2022] DELETE /identity/v3/projects/c209de842bf348dcbdc1b592630b51b5 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:06.609974 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc38536-17b7-40da-8a01-48654c58d8ec None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:06.610635 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc38536-17b7-40da-8a01-48654c58d8ec None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:06.610739 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc38536-17b7-40da-8a01-48654c58d8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:06.631792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-3dc38536-17b7-40da-8a01-48654c58d8ec None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:06.633012 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-3dc38536-17b7-40da-8a01-48654c58d8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:06.693227 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1253/2519] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:06.703516 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99925b-c495-461c-b862-d8bd5e8db1f1 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:06.703795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99925b-c495-461c-b862-d8bd5e8db1f1 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:06.704193 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99925b-c495-461c-b862-d8bd5e8db1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:06.734887 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-db99925b-c495-461c-b862-d8bd5e8db1f1 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:06.736224 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-db99925b-c495-461c-b862-d8bd5e8db1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:06.745900 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1267/2520] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:06.751132 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc38536-17b7-40da-8a01-48654c58d8ec None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:06.766423 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc38536-17b7-40da-8a01-48654c58d8ec None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:06.772469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4a553-7cd3-44b4-adf0-09499ed5202e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:06.773098 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4a553-7cd3-44b4-adf0-09499ed5202e admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:06.773402 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4a553-7cd3-44b4-adf0-09499ed5202e admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:06.773668 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4a553-7cd3-44b4-adf0-09499ed5202e admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:06.774310 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4a553-7cd3-44b4-adf0-09499ed5202e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:19:06.774605 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4a553-7cd3-44b4-adf0-09499ed5202e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:06.776222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4a553-7cd3-44b4-adf0-09499ed5202e admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:06.788726 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1254/2521] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:19:06.797008 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db99925b-c495-461c-b862-d8bd5e8db1f1 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:06.807667 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db99925b-c495-461c-b862-d8bd5e8db1f1 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:06.813740 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ba5f1-467d-4c47-93b0-f7a76380b011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:06.814684 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ba5f1-467d-4c47-93b0-f7a76380b011 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:06.814958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ba5f1-467d-4c47-93b0-f7a76380b011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:06.815234 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ba5f1-467d-4c47-93b0-f7a76380b011 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:06.815873 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ba5f1-467d-4c47-93b0-f7a76380b011 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:06.817830 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ba5f1-467d-4c47-93b0-f7a76380b011 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:06.856348 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1268/2522] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:19:06 2022] POST /identity/v3/projects => generated 389 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:19:06.862668 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4a553-7cd3-44b4-adf0-09499ed5202e admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:06.873118 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4a553-7cd3-44b4-adf0-09499ed5202e admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:06.935751 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83631864-7035-43e2-babf-9e421f5d27dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:06.936637 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83631864-7035-43e2-babf-9e421f5d27dd admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:06.937015 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83631864-7035-43e2-babf-9e421f5d27dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:06.937486 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83631864-7035-43e2-babf-9e421f5d27dd admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:06.938496 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83631864-7035-43e2-babf-9e421f5d27dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:06.941098 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83631864-7035-43e2-babf-9e421f5d27dd admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:06.969554 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1255/2523] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:19:06 2022] POST /identity/v3/users => generated 295 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:19:06.974844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ba5f1-467d-4c47-93b0-f7a76380b011 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:06.987847 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ba5f1-467d-4c47-93b0-f7a76380b011 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:06.994265 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb5e5b-8005-4632-ad7a-df7ec8c2b799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:06.995094 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb5e5b-8005-4632-ad7a-df7ec8c2b799 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:06.995419 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb5e5b-8005-4632-ad7a-df7ec8c2b799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:06.995753 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb5e5b-8005-4632-ad7a-df7ec8c2b799 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:06.996696 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb5e5b-8005-4632-ad7a-df7ec8c2b799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:06.998971 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb5e5b-8005-4632-ad7a-df7ec8c2b799 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:07.011918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1269/2524] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:19:07.018003 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83631864-7035-43e2-babf-9e421f5d27dd admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:07.028922 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83631864-7035-43e2-babf-9e421f5d27dd admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:07.034475 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e03fc-00a0-4775-a16f-1b1713d17d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:07.035080 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e03fc-00a0-4775-a16f-1b1713d17d25 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:07.035351 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e03fc-00a0-4775-a16f-1b1713d17d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:07.035598 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e03fc-00a0-4775-a16f-1b1713d17d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:07.036230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e03fc-00a0-4775-a16f-1b1713d17d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:07.037932 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e03fc-00a0-4775-a16f-1b1713d17d25 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:07.049678 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1256/2525] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:19:07.053816 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb5e5b-8005-4632-ad7a-df7ec8c2b799 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:07.064976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb5e5b-8005-4632-ad7a-df7ec8c2b799 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:07.070269 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102f4c3-6e97-4a7f-a1c5-e56dd7348946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:07.070918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102f4c3-6e97-4a7f-a1c5-e56dd7348946 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:07.071195 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102f4c3-6e97-4a7f-a1c5-e56dd7348946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:07.071467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102f4c3-6e97-4a7f-a1c5-e56dd7348946 admin admin] PATH_INFO: `/v3/projects/48ff63508bd84ebb800df9805204641a/users/600215239db040a9951ee99ffc45d517/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:07.088643 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d102f4c3-6e97-4a7f-a1c5-e56dd7348946 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48ff63508bd84ebb800df9805204641a, user_id=600215239db040a9951ee99ffc45d517, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:07.091349 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d102f4c3-6e97-4a7f-a1c5-e56dd7348946 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:07.107488 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1270/2526] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:19:07 2022] PUT /identity/v3/projects/48ff63508bd84ebb800df9805204641a/users/600215239db040a9951ee99ffc45d517/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:07.119352 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ba5db-1a77-497e-bb30-1316673379ce None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:07.119630 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ba5db-1a77-497e-bb30-1316673379ce None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:07.119897 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ba5db-1a77-497e-bb30-1316673379ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:07.148360 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-c92ba5db-1a77-497e-bb30-1316673379ce None None] MFA Rules not processed for user `600215239db040a9951ee99ffc45d517`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:07.149628 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c92ba5db-1a77-497e-bb30-1316673379ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:07.189555 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:07.221107 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1257/2527] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:07 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:07.226164 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102f4c3-6e97-4a7f-a1c5-e56dd7348946 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:07.293961 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102f4c3-6e97-4a7f-a1c5-e56dd7348946 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:07.299076 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c153a75-bb02-44da-b6cf-33cfde423ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:07.299682 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c153a75-bb02-44da-b6cf-33cfde423ec9 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:07.299959 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c153a75-bb02-44da-b6cf-33cfde423ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:07.300228 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c153a75-bb02-44da-b6cf-33cfde423ec9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:07.300860 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c153a75-bb02-44da-b6cf-33cfde423ec9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:07.302815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c153a75-bb02-44da-b6cf-33cfde423ec9 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:07.333197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1271/2528] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:19:07 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:19:07.337538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ba5db-1a77-497e-bb30-1316673379ce None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:07.347979 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ba5db-1a77-497e-bb30-1316673379ce None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:07.405319 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc89907-d065-4b0e-b39f-b9cab9c5b6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:07.405939 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc89907-d065-4b0e-b39f-b9cab9c5b6c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:07.406211 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc89907-d065-4b0e-b39f-b9cab9c5b6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:07.406481 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc89907-d065-4b0e-b39f-b9cab9c5b6c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:07.407133 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc89907-d065-4b0e-b39f-b9cab9c5b6c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:07.409297 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc89907-d065-4b0e-b39f-b9cab9c5b6c3 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:07.440845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1258/2529] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:19:07 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:19:07.441939 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c153a75-bb02-44da-b6cf-33cfde423ec9 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:07.456005 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c153a75-bb02-44da-b6cf-33cfde423ec9 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:07.462014 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842ada1-78d8-4558-9478-eabc2c40ac42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:07.462659 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842ada1-78d8-4558-9478-eabc2c40ac42 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:07.462937 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842ada1-78d8-4558-9478-eabc2c40ac42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:07.463268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842ada1-78d8-4558-9478-eabc2c40ac42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:07.463969 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842ada1-78d8-4558-9478-eabc2c40ac42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:07.465834 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842ada1-78d8-4558-9478-eabc2c40ac42 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:07.479247 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1272/2530] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:19:07.483696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc89907-d065-4b0e-b39f-b9cab9c5b6c3 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:07.494362 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc89907-d065-4b0e-b39f-b9cab9c5b6c3 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:07.499159 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3730299-9fc4-48ec-b472-3d9180a0d809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:07.499890 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3730299-9fc4-48ec-b472-3d9180a0d809 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:07.500230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3730299-9fc4-48ec-b472-3d9180a0d809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:07.500558 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3730299-9fc4-48ec-b472-3d9180a0d809 admin admin] PATH_INFO: `/v3/projects/e24595c0160b4559b1d1bc858448af3d/users/95926ade7a3e437eb3c4e0453de9bda0/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:07.515478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3730299-9fc4-48ec-b472-3d9180a0d809 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e24595c0160b4559b1d1bc858448af3d, user_id=95926ade7a3e437eb3c4e0453de9bda0, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:07.518258 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3730299-9fc4-48ec-b472-3d9180a0d809 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:07.534643 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1259/2531] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:19:07 2022] PUT /identity/v3/projects/e24595c0160b4559b1d1bc858448af3d/users/95926ade7a3e437eb3c4e0453de9bda0/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:07.538741 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842ada1-78d8-4558-9478-eabc2c40ac42 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:07.549325 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842ada1-78d8-4558-9478-eabc2c40ac42 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:07.554018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:07.608030 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25f921-4eae-460f-ba77-b11f6e86627d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:07.608632 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25f921-4eae-460f-ba77-b11f6e86627d admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:07.608881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25f921-4eae-460f-ba77-b11f6e86627d admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:07.609163 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25f921-4eae-460f-ba77-b11f6e86627d admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:07.609840 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25f921-4eae-460f-ba77-b11f6e86627d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:07.611484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25f921-4eae-460f-ba77-b11f6e86627d admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:07.621342 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1273/2532] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:19:07.626299 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3730299-9fc4-48ec-b472-3d9180a0d809 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:07.637660 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3730299-9fc4-48ec-b472-3d9180a0d809 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:07.643325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfce5a61-e127-476a-a283-17db5ae0223c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:07.643964 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce5a61-e127-476a-a283-17db5ae0223c admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:07.644234 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce5a61-e127-476a-a283-17db5ae0223c admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:07.644496 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce5a61-e127-476a-a283-17db5ae0223c admin admin] PATH_INFO: `/v3/projects/e24595c0160b4559b1d1bc858448af3d/users/95926ade7a3e437eb3c4e0453de9bda0/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:07.646718 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfce5a61-e127-476a-a283-17db5ae0223c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e24595c0160b4559b1d1bc858448af3d, user_id=95926ade7a3e437eb3c4e0453de9bda0, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:07.649389 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfce5a61-e127-476a-a283-17db5ae0223c admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:07.660948 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1260/2533] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:19:07 2022] PUT /identity/v3/projects/e24595c0160b4559b1d1bc858448af3d/users/95926ade7a3e437eb3c4e0453de9bda0/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:07.671634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c6984-8141-426e-85c4-53cfaf7cf2d9 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:07.671920 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c6984-8141-426e-85c4-53cfaf7cf2d9 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:07.672190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c6984-8141-426e-85c4-53cfaf7cf2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:07.702609 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-751c6984-8141-426e-85c4-53cfaf7cf2d9 None None] MFA Rules not processed for user `95926ade7a3e437eb3c4e0453de9bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:07.703891 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-751c6984-8141-426e-85c4-53cfaf7cf2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:07.774906 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1274/2534] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:07 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:07.787223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfce5a61-e127-476a-a283-17db5ae0223c admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:07.789095 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-bfce5a61-e127-476a-a283-17db5ae0223c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:07.852482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfce5a61-e127-476a-a283-17db5ae0223c admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:07.856427 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e3528f-5125-4039-b5cc-3602705112ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:07.857036 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e3528f-5125-4039-b5cc-3602705112ec service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:07.857347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e3528f-5125-4039-b5cc-3602705112ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:07.857617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e3528f-5125-4039-b5cc-3602705112ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:07.868311 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e3528f-5125-4039-b5cc-3602705112ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:07.869797 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e3528f-5125-4039-b5cc-3602705112ec service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:07.880661 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1261/2535] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:19:07 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:19:08.024478 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c6984-8141-426e-85c4-53cfaf7cf2d9 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:08.109245 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c6984-8141-426e-85c4-53cfaf7cf2d9 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:08.117565 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5df04b-6711-4a55-9a9b-b22cc66803d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:08.118443 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5df04b-6711-4a55-9a9b-b22cc66803d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:08.118803 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5df04b-6711-4a55-9a9b-b22cc66803d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:08.119167 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5df04b-6711-4a55-9a9b-b22cc66803d2 admin admin] PATH_INFO: `/v3/users/600215239db040a9951ee99ffc45d517` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:08.121101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5df04b-6711-4a55-9a9b-b22cc66803d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=600215239db040a9951ee99ffc45d517)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:08.124064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5df04b-6711-4a55-9a9b-b22cc66803d2 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:08.169490 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:08.182406 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-1a5df04b-6711-4a55-9a9b-b22cc66803d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '600215239db040a9951ee99ffc45d517'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:08.191059 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-1a5df04b-6711-4a55-9a9b-b22cc66803d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '600215239db040a9951ee99ffc45d517'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:08.203486 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-1a5df04b-6711-4a55-9a9b-b22cc66803d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '600215239db040a9951ee99ffc45d517'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:08.204180 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-1a5df04b-6711-4a55-9a9b-b22cc66803d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '600215239db040a9951ee99ffc45d517'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:08.211077 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-1a5df04b-6711-4a55-9a9b-b22cc66803d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '600215239db040a9951ee99ffc45d517'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:08.216936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1275/2536] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:19:08 2022] DELETE /identity/v3/users/600215239db040a9951ee99ffc45d517 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:08.221624 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd8f86-90d2-4962-a0c5-9d68dc0d9ea2 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:08.221898 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd8f86-90d2-4962-a0c5-9d68dc0d9ea2 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:08.222165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd8f86-90d2-4962-a0c5-9d68dc0d9ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:08.241696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-36dd8f86-90d2-4962-a0c5-9d68dc0d9ea2 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:08.242884 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-36dd8f86-90d2-4962-a0c5-9d68dc0d9ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:08.273726 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:08.306564 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1262/2537] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:08.317552 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5df04b-6711-4a55-9a9b-b22cc66803d2 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:08.319007 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-1a5df04b-6711-4a55-9a9b-b22cc66803d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:08.388475 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5df04b-6711-4a55-9a9b-b22cc66803d2 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:08.392202 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8cc94-5e1a-4e8a-a492-f2d04ddc9bdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:08.392873 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8cc94-5e1a-4e8a-a492-f2d04ddc9bdf service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:08.393157 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8cc94-5e1a-4e8a-a492-f2d04ddc9bdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:08.393431 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8cc94-5e1a-4e8a-a492-f2d04ddc9bdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:08.403265 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8cc94-5e1a-4e8a-a492-f2d04ddc9bdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:08.404694 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d8cc94-5e1a-4e8a-a492-f2d04ddc9bdf service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:08.414650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1276/2538] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:19:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:19:08.686195 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd8f86-90d2-4962-a0c5-9d68dc0d9ea2 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:08.687431 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-36dd8f86-90d2-4962-a0c5-9d68dc0d9ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:08.701443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd8f86-90d2-4962-a0c5-9d68dc0d9ea2 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:08.706603 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b5569-4713-42df-9763-ddf528b0520d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:08.707255 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b5569-4713-42df-9763-ddf528b0520d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:08.707530 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b5569-4713-42df-9763-ddf528b0520d admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:08.707801 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b5569-4713-42df-9763-ddf528b0520d admin admin] PATH_INFO: `/v3/projects/48ff63508bd84ebb800df9805204641a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:08.715878 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b5569-4713-42df-9763-ddf528b0520d admin admin] RBAC: Authorizing `identity:delete_project(project_id=48ff63508bd84ebb800df9805204641a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:08.717875 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b5569-4713-42df-9763-ddf528b0520d admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:08.770003 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-1b2b5569-4713-42df-9763-ddf528b0520d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48ff63508bd84ebb800df9805204641a', 'request_id': 'req-1b2b5569-4713-42df-9763-ddf528b0520d'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:08.780207 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-1b2b5569-4713-42df-9763-ddf528b0520d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48ff63508bd84ebb800df9805204641a', 'request_id': 'req-1b2b5569-4713-42df-9763-ddf528b0520d'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:08.787956 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-1b2b5569-4713-42df-9763-ddf528b0520d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48ff63508bd84ebb800df9805204641a', 'request_id': 'req-1b2b5569-4713-42df-9763-ddf528b0520d'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:08.802005 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-1b2b5569-4713-42df-9763-ddf528b0520d admin admin] The token cache is being invalidate because project 48ff63508bd84ebb800df9805204641a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:19:08.802284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-1b2b5569-4713-42df-9763-ddf528b0520d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:08.802935 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-1b2b5569-4713-42df-9763-ddf528b0520d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:08.804444 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1263/2539] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:19:08 2022] DELETE /identity/v3/projects/48ff63508bd84ebb800df9805204641a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:08.808572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8cc94-5e1a-4e8a-a492-f2d04ddc9bdf service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:08.818828 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d8cc94-5e1a-4e8a-a492-f2d04ddc9bdf service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:08.880365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48af5d55-bd19-491f-b990-44a63df5d68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:08.880996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af5d55-bd19-491f-b990-44a63df5d68a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:08.881305 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af5d55-bd19-491f-b990-44a63df5d68a admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:08.881575 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af5d55-bd19-491f-b990-44a63df5d68a admin admin] PATH_INFO: `/v3/users/95926ade7a3e437eb3c4e0453de9bda0` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:08.882723 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48af5d55-bd19-491f-b990-44a63df5d68a admin admin] RBAC: Authorizing `identity:delete_user(user_id=95926ade7a3e437eb3c4e0453de9bda0)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:08.884635 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48af5d55-bd19-491f-b990-44a63df5d68a admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:08.934082 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-48af5d55-bd19-491f-b990-44a63df5d68a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95926ade7a3e437eb3c4e0453de9bda0'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:08.944334 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-48af5d55-bd19-491f-b990-44a63df5d68a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95926ade7a3e437eb3c4e0453de9bda0'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:08.945774 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:08.961589 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-48af5d55-bd19-491f-b990-44a63df5d68a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95926ade7a3e437eb3c4e0453de9bda0'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:08.962360 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-48af5d55-bd19-491f-b990-44a63df5d68a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95926ade7a3e437eb3c4e0453de9bda0'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:08.969897 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-48af5d55-bd19-491f-b990-44a63df5d68a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95926ade7a3e437eb3c4e0453de9bda0'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:08.976552 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1277/2540] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:19:08 2022] DELETE /identity/v3/users/95926ade7a3e437eb3c4e0453de9bda0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:09.276252 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b5569-4713-42df-9763-ddf528b0520d admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:09.277859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-1b2b5569-4713-42df-9763-ddf528b0520d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:09.292613 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b5569-4713-42df-9763-ddf528b0520d admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:09.350912 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585e936-2461-4399-b52b-bfe9822568e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:09.351544 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585e936-2461-4399-b52b-bfe9822568e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:09.351823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585e936-2461-4399-b52b-bfe9822568e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:09.352087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585e936-2461-4399-b52b-bfe9822568e2 admin admin] PATH_INFO: `/v3/projects/e24595c0160b4559b1d1bc858448af3d` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:09.353215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e585e936-2461-4399-b52b-bfe9822568e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e24595c0160b4559b1d1bc858448af3d)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:09.355126 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e585e936-2461-4399-b52b-bfe9822568e2 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:09.404695 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-e585e936-2461-4399-b52b-bfe9822568e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e24595c0160b4559b1d1bc858448af3d', 'request_id': 'req-e585e936-2461-4399-b52b-bfe9822568e2'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:09.413985 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-e585e936-2461-4399-b52b-bfe9822568e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e24595c0160b4559b1d1bc858448af3d', 'request_id': 'req-e585e936-2461-4399-b52b-bfe9822568e2'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:09.416274 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:09.421416 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-e585e936-2461-4399-b52b-bfe9822568e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e24595c0160b4559b1d1bc858448af3d', 'request_id': 'req-e585e936-2461-4399-b52b-bfe9822568e2'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:09.433728 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-e585e936-2461-4399-b52b-bfe9822568e2 admin admin] The token cache is being invalidate because project e24595c0160b4559b1d1bc858448af3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:19:09.434009 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-e585e936-2461-4399-b52b-bfe9822568e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:09.434635 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-e585e936-2461-4399-b52b-bfe9822568e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:09.436175 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1264/2541] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:19:09 2022] DELETE /identity/v3/projects/e24595c0160b4559b1d1bc858448af3d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:09.441713 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435779b-d6f2-4d5d-bfc1-6a2a2c900d31 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:09.442000 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435779b-d6f2-4d5d-bfc1-6a2a2c900d31 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:09.442266 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435779b-d6f2-4d5d-bfc1-6a2a2c900d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:09.462960 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-f435779b-d6f2-4d5d-bfc1-6a2a2c900d31 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:09.464090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-f435779b-d6f2-4d5d-bfc1-6a2a2c900d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:09.528951 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1278/2542] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:09.537783 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5b215-641b-4f3c-9b63-9b71a674d192 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:09.538144 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5b215-641b-4f3c-9b63-9b71a674d192 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:09.538500 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5b215-641b-4f3c-9b63-9b71a674d192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:09.562861 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-c9b5b215-641b-4f3c-9b63-9b71a674d192 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:09.564085 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c9b5b215-641b-4f3c-9b63-9b71a674d192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:09.573531 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1265/2543] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:09.578102 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435779b-d6f2-4d5d-bfc1-6a2a2c900d31 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:09.587954 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435779b-d6f2-4d5d-bfc1-6a2a2c900d31 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:09.593283 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb7c554-1730-4a81-8146-ba5389096554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:09.593887 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7c554-1730-4a81-8146-ba5389096554 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:09.594159 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7c554-1730-4a81-8146-ba5389096554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:09.594429 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7c554-1730-4a81-8146-ba5389096554 admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:09.595113 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb7c554-1730-4a81-8146-ba5389096554 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:19:09.595411 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb7c554-1730-4a81-8146-ba5389096554 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:09.597006 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb7c554-1730-4a81-8146-ba5389096554 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:09.608707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1279/2544] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:19:09.613422 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5b215-641b-4f3c-9b63-9b71a674d192 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:09.625390 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5b215-641b-4f3c-9b63-9b71a674d192 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:09.632868 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e936690-9d61-42d0-bd59-afa3be1511b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:09.633764 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e936690-9d61-42d0-bd59-afa3be1511b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:09.634106 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e936690-9d61-42d0-bd59-afa3be1511b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:09.634427 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e936690-9d61-42d0-bd59-afa3be1511b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:09.635371 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e936690-9d61-42d0-bd59-afa3be1511b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:09.638200 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e936690-9d61-42d0-bd59-afa3be1511b0 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:09.670028 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1266/2545] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:19:09 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:19:09.674299 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb7c554-1730-4a81-8146-ba5389096554 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:09.684516 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb7c554-1730-4a81-8146-ba5389096554 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:09.743449 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf46eb-1ca3-41be-a73a-eed625e92426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:09.743974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf46eb-1ca3-41be-a73a-eed625e92426 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:09.744236 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf46eb-1ca3-41be-a73a-eed625e92426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:09.744528 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf46eb-1ca3-41be-a73a-eed625e92426 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:09.745227 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf46eb-1ca3-41be-a73a-eed625e92426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:09.747147 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf46eb-1ca3-41be-a73a-eed625e92426 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:09.771155 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1280/2546] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:19:09 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:19:09.775485 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e936690-9d61-42d0-bd59-afa3be1511b0 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:09.786657 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e936690-9d61-42d0-bd59-afa3be1511b0 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:09.791867 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd2fea-c2ad-4a80-97e2-006433303430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:09.792541 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd2fea-c2ad-4a80-97e2-006433303430 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:09.792816 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd2fea-c2ad-4a80-97e2-006433303430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:09.793082 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd2fea-c2ad-4a80-97e2-006433303430 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:09.793760 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd2fea-c2ad-4a80-97e2-006433303430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:09.795384 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd2fea-c2ad-4a80-97e2-006433303430 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:09.808201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1267/2547] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:19:09.811302 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf46eb-1ca3-41be-a73a-eed625e92426 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:09.822130 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf46eb-1ca3-41be-a73a-eed625e92426 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:09.826839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47bebd-06a8-41ab-a884-df142a38bb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:09.827450 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47bebd-06a8-41ab-a884-df142a38bb03 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:09.827866 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47bebd-06a8-41ab-a884-df142a38bb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:09.828136 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47bebd-06a8-41ab-a884-df142a38bb03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:09.828810 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a47bebd-06a8-41ab-a884-df142a38bb03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:09.830511 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a47bebd-06a8-41ab-a884-df142a38bb03 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:09.841359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1281/2548] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:19:09.845580 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd2fea-c2ad-4a80-97e2-006433303430 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:09.855914 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd2fea-c2ad-4a80-97e2-006433303430 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:09.865512 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4051728-eaa1-4946-b874-4f0468ac849a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:09.866125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4051728-eaa1-4946-b874-4f0468ac849a admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:09.866394 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4051728-eaa1-4946-b874-4f0468ac849a admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:09.866665 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4051728-eaa1-4946-b874-4f0468ac849a admin admin] PATH_INFO: `/v3/projects/0a6b6af01f444c299896cc72107dc6dc/users/5beef593ebc84973a519d486edf6ecdc/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:09.882989 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4051728-eaa1-4946-b874-4f0468ac849a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a6b6af01f444c299896cc72107dc6dc, user_id=5beef593ebc84973a519d486edf6ecdc, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:09.885697 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4051728-eaa1-4946-b874-4f0468ac849a admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:09.900430 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1268/2549] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:19:09 2022] PUT /identity/v3/projects/0a6b6af01f444c299896cc72107dc6dc/users/5beef593ebc84973a519d486edf6ecdc/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:09.912639 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800d9f9-749c-4e17-b1c2-e25b6424383b None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:09.912969 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800d9f9-749c-4e17-b1c2-e25b6424383b None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:09.913387 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800d9f9-749c-4e17-b1c2-e25b6424383b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:09.950909 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-c800d9f9-749c-4e17-b1c2-e25b6424383b None None] MFA Rules not processed for user `5beef593ebc84973a519d486edf6ecdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:09.952129 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-c800d9f9-749c-4e17-b1c2-e25b6424383b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:09.981643 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:10.027224 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1282/2550] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:19:10.039525 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4051728-eaa1-4946-b874-4f0468ac849a admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:10.041477 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f4051728-eaa1-4946-b874-4f0468ac849a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:10.102641 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4051728-eaa1-4946-b874-4f0468ac849a admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:10.106832 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4780388-d5d1-4af5-a552-1ac8a7c6bc15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:10.107422 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4780388-d5d1-4af5-a552-1ac8a7c6bc15 service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:10.107696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4780388-d5d1-4af5-a552-1ac8a7c6bc15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:10.107960 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4780388-d5d1-4af5-a552-1ac8a7c6bc15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:10.117519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4780388-d5d1-4af5-a552-1ac8a7c6bc15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:10.119062 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4780388-d5d1-4af5-a552-1ac8a7c6bc15 service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:10.128517 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1269/2551] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:19:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:19:20.024279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=15 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:20.123969 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=58,keystone:DELETE=1,keystone:INSERT=2 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:26.123516 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800d9f9-749c-4e17-b1c2-e25b6424383b None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:26.125059 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-c800d9f9-749c-4e17-b1c2-e25b6424383b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:26.192101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800d9f9-749c-4e17-b1c2-e25b6424383b None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:26.196356 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfe245-f464-4439-a0ad-f9bc6dd0f9dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:26.196985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfe245-f464-4439-a0ad-f9bc6dd0f9dc service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:26.197286 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfe245-f464-4439-a0ad-f9bc6dd0f9dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:26.197558 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfe245-f464-4439-a0ad-f9bc6dd0f9dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:26.199642 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-6bdfe245-f464-4439-a0ad-f9bc6dd0f9dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:26.285052 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdfe245-f464-4439-a0ad-f9bc6dd0f9dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:26.286564 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdfe245-f464-4439-a0ad-f9bc6dd0f9dc service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:26.297890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1283/2552] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:19:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:19:28.198767 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4780388-d5d1-4af5-a552-1ac8a7c6bc15 service cinder] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:28.215651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4780388-d5d1-4af5-a552-1ac8a7c6bc15 service cinder] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:28.222026 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f82c7-d56a-4002-a032-fc098c74c49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:28.223067 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f82c7-d56a-4002-a032-fc098c74c49c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:28.223520 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f82c7-d56a-4002-a032-fc098c74c49c admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:28.223972 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f82c7-d56a-4002-a032-fc098c74c49c admin admin] PATH_INFO: `/v3/users/5beef593ebc84973a519d486edf6ecdc` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:28.225541 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f82c7-d56a-4002-a032-fc098c74c49c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5beef593ebc84973a519d486edf6ecdc)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:28.227523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f82c7-d56a-4002-a032-fc098c74c49c admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:28.297328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-785f82c7-d56a-4002-a032-fc098c74c49c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5beef593ebc84973a519d486edf6ecdc'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:28.310597 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-785f82c7-d56a-4002-a032-fc098c74c49c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5beef593ebc84973a519d486edf6ecdc'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:28.324735 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-785f82c7-d56a-4002-a032-fc098c74c49c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5beef593ebc84973a519d486edf6ecdc'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:28.325669 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-785f82c7-d56a-4002-a032-fc098c74c49c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5beef593ebc84973a519d486edf6ecdc'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:28.334056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-785f82c7-d56a-4002-a032-fc098c74c49c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5beef593ebc84973a519d486edf6ecdc'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:28.340764 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1270/2553] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:19:28 2022] DELETE /identity/v3/users/5beef593ebc84973a519d486edf6ecdc => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:28.345589 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34c626-f5b2-403a-baba-99310d5692fd None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:28.346118 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34c626-f5b2-403a-baba-99310d5692fd None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:28.346227 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34c626-f5b2-403a-baba-99310d5692fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:28.368682 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-7f34c626-f5b2-403a-baba-99310d5692fd None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:28.370542 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-7f34c626-f5b2-403a-baba-99310d5692fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:28.470650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1284/2554] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:28.482661 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f82c7-d56a-4002-a032-fc098c74c49c admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:28.485176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-785f82c7-d56a-4002-a032-fc098c74c49c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:28.552513 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f82c7-d56a-4002-a032-fc098c74c49c admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:28.556616 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0478c775-1771-40f1-a966-dfc01e296546 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:28.557405 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478c775-1771-40f1-a966-dfc01e296546 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:28.558409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478c775-1771-40f1-a966-dfc01e296546 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:28.558409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478c775-1771-40f1-a966-dfc01e296546 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:28.569355 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0478c775-1771-40f1-a966-dfc01e296546 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:28.570811 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0478c775-1771-40f1-a966-dfc01e296546 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:28.582251 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1271/2555] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:19:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:19:28.995188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34c626-f5b2-403a-baba-99310d5692fd None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:28.997427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-7f34c626-f5b2-403a-baba-99310d5692fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:29.010573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34c626-f5b2-403a-baba-99310d5692fd None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:29.016733 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9685b-5c5d-404b-b495-2842289616f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:29.017462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9685b-5c5d-404b-b495-2842289616f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:29.017742 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9685b-5c5d-404b-b495-2842289616f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:29.018011 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9685b-5c5d-404b-b495-2842289616f9 admin admin] PATH_INFO: `/v3/projects/0a6b6af01f444c299896cc72107dc6dc` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:29.019285 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e9685b-5c5d-404b-b495-2842289616f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a6b6af01f444c299896cc72107dc6dc)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:29.021266 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e9685b-5c5d-404b-b495-2842289616f9 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:29.089734 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:29.093462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e6e9685b-5c5d-404b-b495-2842289616f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a6b6af01f444c299896cc72107dc6dc', 'request_id': 'req-e6e9685b-5c5d-404b-b495-2842289616f9'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:29.100031 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e6e9685b-5c5d-404b-b495-2842289616f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a6b6af01f444c299896cc72107dc6dc', 'request_id': 'req-e6e9685b-5c5d-404b-b495-2842289616f9'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:29.106884 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e6e9685b-5c5d-404b-b495-2842289616f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a6b6af01f444c299896cc72107dc6dc', 'request_id': 'req-e6e9685b-5c5d-404b-b495-2842289616f9'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:29.121336 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e6e9685b-5c5d-404b-b495-2842289616f9 admin admin] The token cache is being invalidate because project 0a6b6af01f444c299896cc72107dc6dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:19:29.121621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e6e9685b-5c5d-404b-b495-2842289616f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:29.122312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-e6e9685b-5c5d-404b-b495-2842289616f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:29.123833 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1285/2556] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:19:28 2022] DELETE /identity/v3/projects/0a6b6af01f444c299896cc72107dc6dc => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:29.129126 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb08e1-c73e-4334-a0af-4c464b037ec9 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:29.129684 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb08e1-c73e-4334-a0af-4c464b037ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:29.129935 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb08e1-c73e-4334-a0af-4c464b037ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:29.168553 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-29eb08e1-c73e-4334-a0af-4c464b037ec9 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:29.169920 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-29eb08e1-c73e-4334-a0af-4c464b037ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:29.260157 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1272/2557] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:29.271997 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc519fd-1bb6-4100-9a12-d9f9c809f751 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:29.273116 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc519fd-1bb6-4100-9a12-d9f9c809f751 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:29.273843 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc519fd-1bb6-4100-9a12-d9f9c809f751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:29.310187 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-2cc519fd-1bb6-4100-9a12-d9f9c809f751 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:29.311854 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-2cc519fd-1bb6-4100-9a12-d9f9c809f751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:29.325174 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1286/2558] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:29.328793 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb08e1-c73e-4334-a0af-4c464b037ec9 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:29.348841 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb08e1-c73e-4334-a0af-4c464b037ec9 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:29.355413 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c6d820-2c71-49d5-b22b-70c9a03b9b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:29.356265 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6d820-2c71-49d5-b22b-70c9a03b9b33 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:29.356734 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6d820-2c71-49d5-b22b-70c9a03b9b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:29.357222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6d820-2c71-49d5-b22b-70c9a03b9b33 admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:29.358054 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c6d820-2c71-49d5-b22b-70c9a03b9b33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:19:29.358509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c6d820-2c71-49d5-b22b-70c9a03b9b33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:29.360334 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c6d820-2c71-49d5-b22b-70c9a03b9b33 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:29.380403 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1273/2559] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:19:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:19:29.386093 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc519fd-1bb6-4100-9a12-d9f9c809f751 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:29.401684 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc519fd-1bb6-4100-9a12-d9f9c809f751 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:29.415444 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639def06-e2a7-48aa-9e90-0135860179dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:29.416098 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639def06-e2a7-48aa-9e90-0135860179dc admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:29.416376 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639def06-e2a7-48aa-9e90-0135860179dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:29.416647 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639def06-e2a7-48aa-9e90-0135860179dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:29.417680 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639def06-e2a7-48aa-9e90-0135860179dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:29.420545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639def06-e2a7-48aa-9e90-0135860179dc admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:29.462571 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1287/2560] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:19:29 2022] POST /identity/v3/projects => generated 399 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:19:29.467067 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c6d820-2c71-49d5-b22b-70c9a03b9b33 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:29.480864 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:29.494726 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c6d820-2c71-49d5-b22b-70c9a03b9b33 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:29.583991 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f04d081-e93c-4543-91f5-7682b87c4c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:29.584888 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04d081-e93c-4543-91f5-7682b87c4c0d admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:29.585326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04d081-e93c-4543-91f5-7682b87c4c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:29.585696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04d081-e93c-4543-91f5-7682b87c4c0d admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:29.586457 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f04d081-e93c-4543-91f5-7682b87c4c0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:29.589020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f04d081-e93c-4543-91f5-7682b87c4c0d admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:29.623830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1274/2561] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:19:29 2022] POST /identity/v3/users => generated 300 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:19:29.624521 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639def06-e2a7-48aa-9e90-0135860179dc admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:29.639675 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639def06-e2a7-48aa-9e90-0135860179dc admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:29.648151 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d65d2-f694-4bd7-9f76-6e0e70d0f03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:29.648839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d65d2-f694-4bd7-9f76-6e0e70d0f03b admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:29.649187 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d65d2-f694-4bd7-9f76-6e0e70d0f03b admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:29.649470 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d65d2-f694-4bd7-9f76-6e0e70d0f03b admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:29.650141 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1d65d2-f694-4bd7-9f76-6e0e70d0f03b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:29.651805 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1d65d2-f694-4bd7-9f76-6e0e70d0f03b admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:29.676727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1288/2562] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:19:29.681021 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f04d081-e93c-4543-91f5-7682b87c4c0d admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:29.691985 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f04d081-e93c-4543-91f5-7682b87c4c0d admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:29.696858 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe60ee-bc82-4676-8807-ee5707d2ae83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:29.700028 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe60ee-bc82-4676-8807-ee5707d2ae83 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:29.700028 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe60ee-bc82-4676-8807-ee5707d2ae83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:29.700028 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe60ee-bc82-4676-8807-ee5707d2ae83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:29.700028 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe60ee-bc82-4676-8807-ee5707d2ae83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:29.701065 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe60ee-bc82-4676-8807-ee5707d2ae83 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:29.719011 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1275/2563] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:19:29.720719 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d65d2-f694-4bd7-9f76-6e0e70d0f03b admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:29.737835 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d65d2-f694-4bd7-9f76-6e0e70d0f03b admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:29.751795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18aae88-5abc-4e10-84c7-ac2d7b9b53b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:29.752858 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18aae88-5abc-4e10-84c7-ac2d7b9b53b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:29.753231 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18aae88-5abc-4e10-84c7-ac2d7b9b53b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:29.753492 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18aae88-5abc-4e10-84c7-ac2d7b9b53b3 admin admin] PATH_INFO: `/v3/projects/0d2d462ebe4c477b9712e391bfb85d96/users/d1d86083b55244b8948cd1e007d3829e/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:29.770089 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe60ee-bc82-4676-8807-ee5707d2ae83 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:29.772071 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-9abe60ee-bc82-4676-8807-ee5707d2ae83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:29.777070 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18aae88-5abc-4e10-84c7-ac2d7b9b53b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d2d462ebe4c477b9712e391bfb85d96, user_id=d1d86083b55244b8948cd1e007d3829e, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:29.781020 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18aae88-5abc-4e10-84c7-ac2d7b9b53b3 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:29.803441 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1289/2564] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:19:29 2022] PUT /identity/v3/projects/0d2d462ebe4c477b9712e391bfb85d96/users/d1d86083b55244b8948cd1e007d3829e/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:19:29.815594 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c3705-bcf7-482f-8d35-2f3ea89ad0fe None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:29.816309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c3705-bcf7-482f-8d35-2f3ea89ad0fe None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:29.819584 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c3705-bcf7-482f-8d35-2f3ea89ad0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:29.855679 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-741c3705-bcf7-482f-8d35-2f3ea89ad0fe None None] MFA Rules not processed for user `d1d86083b55244b8948cd1e007d3829e`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:29.856888 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-741c3705-bcf7-482f-8d35-2f3ea89ad0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:29.879576 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe60ee-bc82-4676-8807-ee5707d2ae83 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:29.935007 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1290/2565] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:29 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:29.935778 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c3705-bcf7-482f-8d35-2f3ea89ad0fe None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:29.941506 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-741c3705-bcf7-482f-8d35-2f3ea89ad0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:29.952023 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd473cde-67b5-439f-9501-90cd9e506a51 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead06362f8a2444bbfe1405cd602f1c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:29.952650 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd473cde-67b5-439f-9501-90cd9e506a51 service placement] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:29.952928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd473cde-67b5-439f-9501-90cd9e506a51 service placement] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:29.953226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd473cde-67b5-439f-9501-90cd9e506a51 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:29.955763 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-fd473cde-67b5-439f-9501-90cd9e506a51 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:29.971478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd473cde-67b5-439f-9501-90cd9e506a51 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:29.973153 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd473cde-67b5-439f-9501-90cd9e506a51 service placement] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:29.985859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1276/2566] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:19:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:19:29.988578 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd473cde-67b5-439f-9501-90cd9e506a51 service placement] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:29.990141 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-fd473cde-67b5-439f-9501-90cd9e506a51 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:30.010140 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:30.013478 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c3705-bcf7-482f-8d35-2f3ea89ad0fe None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:30.019042 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604b007-584e-4084-80e7-0d054b342795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:30.019712 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604b007-584e-4084-80e7-0d054b342795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:30.019997 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604b007-584e-4084-80e7-0d054b342795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:30.020264 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604b007-584e-4084-80e7-0d054b342795 admin admin] PATH_INFO: `/v3/users/5e4f8b13b66449cf849ff9cb73938c4c` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:30.021582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604b007-584e-4084-80e7-0d054b342795 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e4f8b13b66449cf849ff9cb73938c4c)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:30.023560 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604b007-584e-4084-80e7-0d054b342795 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:30.025824 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:30.065507 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd473cde-67b5-439f-9501-90cd9e506a51 service placement] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:30.070089 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450d3e7-ac42-4624-bf41-5887c8ce0b89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:30.070780 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450d3e7-ac42-4624-bf41-5887c8ce0b89 service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:30.071130 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450d3e7-ac42-4624-bf41-5887c8ce0b89 service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:30.071426 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450d3e7-ac42-4624-bf41-5887c8ce0b89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:30.086897 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c450d3e7-ac42-4624-bf41-5887c8ce0b89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:30.088399 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c450d3e7-ac42-4624-bf41-5887c8ce0b89 service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:30.088654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2604b007-584e-4084-80e7-0d054b342795 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e4f8b13b66449cf849ff9cb73938c4c'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:30.098270 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2604b007-584e-4084-80e7-0d054b342795 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e4f8b13b66449cf849ff9cb73938c4c'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:30.101280 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1277/2567] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:19:29 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:19:30.114002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2604b007-584e-4084-80e7-0d054b342795 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e4f8b13b66449cf849ff9cb73938c4c'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:30.114971 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2604b007-584e-4084-80e7-0d054b342795 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e4f8b13b66449cf849ff9cb73938c4c'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:30.122734 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2604b007-584e-4084-80e7-0d054b342795 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e4f8b13b66449cf849ff9cb73938c4c'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:30.129310 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1291/2568] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:19:29 2022] DELETE /identity/v3/users/5e4f8b13b66449cf849ff9cb73938c4c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:30.361557 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450d3e7-ac42-4624-bf41-5887c8ce0b89 service cinder] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:30.374200 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450d3e7-ac42-4624-bf41-5887c8ce0b89 service cinder] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:30.457790 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7db2d8-6a7a-4944-8020-adcdf9ab2c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:30.458515 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7db2d8-6a7a-4944-8020-adcdf9ab2c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:30.459069 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7db2d8-6a7a-4944-8020-adcdf9ab2c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:30.459069 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7db2d8-6a7a-4944-8020-adcdf9ab2c37 admin admin] PATH_INFO: `/v3/projects/2273d07b98054e3987568fc043597b78` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:30.460427 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7db2d8-6a7a-4944-8020-adcdf9ab2c37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2273d07b98054e3987568fc043597b78)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:30.462443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7db2d8-6a7a-4944-8020-adcdf9ab2c37 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:30.523598 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8f7db2d8-6a7a-4944-8020-adcdf9ab2c37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2273d07b98054e3987568fc043597b78', 'request_id': 'req-8f7db2d8-6a7a-4944-8020-adcdf9ab2c37'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:30.533781 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8f7db2d8-6a7a-4944-8020-adcdf9ab2c37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2273d07b98054e3987568fc043597b78', 'request_id': 'req-8f7db2d8-6a7a-4944-8020-adcdf9ab2c37'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:30.542705 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8f7db2d8-6a7a-4944-8020-adcdf9ab2c37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2273d07b98054e3987568fc043597b78', 'request_id': 'req-8f7db2d8-6a7a-4944-8020-adcdf9ab2c37'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:30.557714 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8f7db2d8-6a7a-4944-8020-adcdf9ab2c37 admin admin] The token cache is being invalidate because project 2273d07b98054e3987568fc043597b78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:19:30.558165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8f7db2d8-6a7a-4944-8020-adcdf9ab2c37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:30.559513 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8f7db2d8-6a7a-4944-8020-adcdf9ab2c37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:30.561841 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1278/2569] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:19:30 2022] DELETE /identity/v3/projects/2273d07b98054e3987568fc043597b78 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:30.568482 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f9aa6-4871-4f22-b87a-4be032ddaec4 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:30.569931 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f9aa6-4871-4f22-b87a-4be032ddaec4 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:30.570431 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f9aa6-4871-4f22-b87a-4be032ddaec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:30.597579 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-b11f9aa6-4871-4f22-b87a-4be032ddaec4 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:30.598890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-b11f9aa6-4871-4f22-b87a-4be032ddaec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:30.678940 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1292/2570] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:30.688442 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e974c2-fa3d-4542-b0d4-7b138eacfacb None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:30.689091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e974c2-fa3d-4542-b0d4-7b138eacfacb None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:30.689477 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e974c2-fa3d-4542-b0d4-7b138eacfacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:30.715349 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-35e974c2-fa3d-4542-b0d4-7b138eacfacb None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:30.716908 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-35e974c2-fa3d-4542-b0d4-7b138eacfacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:30.727050 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1279/2571] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:19:30.731732 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11f9aa6-4871-4f22-b87a-4be032ddaec4 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:30.741939 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11f9aa6-4871-4f22-b87a-4be032ddaec4 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:30.747081 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4980d2-3aad-4422-b3fb-22b0ccf44499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:30.747718 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4980d2-3aad-4422-b3fb-22b0ccf44499 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:30.748024 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4980d2-3aad-4422-b3fb-22b0ccf44499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:30.748295 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4980d2-3aad-4422-b3fb-22b0ccf44499 admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:30.748953 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4980d2-3aad-4422-b3fb-22b0ccf44499 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:19:30.749305 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4980d2-3aad-4422-b3fb-22b0ccf44499 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:30.750943 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4980d2-3aad-4422-b3fb-22b0ccf44499 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:30.763470 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1293/2572] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:19:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:19:30.768149 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e974c2-fa3d-4542-b0d4-7b138eacfacb None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:30.778191 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e974c2-fa3d-4542-b0d4-7b138eacfacb None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:30.783777 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954d2bb-90cb-4bf8-bc6c-fe3cc3075e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:30.784530 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954d2bb-90cb-4bf8-bc6c-fe3cc3075e92 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:30.784877 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954d2bb-90cb-4bf8-bc6c-fe3cc3075e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:30.785396 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954d2bb-90cb-4bf8-bc6c-fe3cc3075e92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:30.786091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954d2bb-90cb-4bf8-bc6c-fe3cc3075e92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:30.788173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954d2bb-90cb-4bf8-bc6c-fe3cc3075e92 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:30.817312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1280/2573] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:19:30 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:19:30.821772 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4980d2-3aad-4422-b3fb-22b0ccf44499 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:30.831952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4980d2-3aad-4422-b3fb-22b0ccf44499 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:30.897439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f36e0-0453-43da-bc74-63fb715d5382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:30.898066 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f36e0-0453-43da-bc74-63fb715d5382 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:30.898373 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f36e0-0453-43da-bc74-63fb715d5382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:30.898645 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f36e0-0453-43da-bc74-63fb715d5382 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:30.899288 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f36e0-0453-43da-bc74-63fb715d5382 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:30.901233 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f36e0-0453-43da-bc74-63fb715d5382 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:30.913940 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:30.926488 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1294/2574] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:19:30 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:19:30.931072 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954d2bb-90cb-4bf8-bc6c-fe3cc3075e92 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:30.942126 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954d2bb-90cb-4bf8-bc6c-fe3cc3075e92 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:30.947231 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ed09c-0e39-4e94-8b6d-e289753a19be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:30.947860 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ed09c-0e39-4e94-8b6d-e289753a19be admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:30.948284 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ed09c-0e39-4e94-8b6d-e289753a19be admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:30.948559 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ed09c-0e39-4e94-8b6d-e289753a19be admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:30.949272 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ed09c-0e39-4e94-8b6d-e289753a19be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:30.950926 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ed09c-0e39-4e94-8b6d-e289753a19be admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:30.963540 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1281/2575] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:19:30.968142 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f36e0-0453-43da-bc74-63fb715d5382 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:30.980256 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f36e0-0453-43da-bc74-63fb715d5382 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:30.988327 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9e4e4-ad5e-484f-b554-36516ae5bd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:30.989290 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9e4e4-ad5e-484f-b554-36516ae5bd0e admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:30.989695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9e4e4-ad5e-484f-b554-36516ae5bd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:30.989975 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9e4e4-ad5e-484f-b554-36516ae5bd0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:30.990648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9e4e4-ad5e-484f-b554-36516ae5bd0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:30.992295 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9e4e4-ad5e-484f-b554-36516ae5bd0e admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:31.004525 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1295/2576] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:19:31.009459 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ed09c-0e39-4e94-8b6d-e289753a19be admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:31.021670 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ed09c-0e39-4e94-8b6d-e289753a19be admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:31.028960 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b6fd4-3fa1-4be5-bc26-2a1b1e69924f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:31.029696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b6fd4-3fa1-4be5-bc26-2a1b1e69924f admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:31.030041 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b6fd4-3fa1-4be5-bc26-2a1b1e69924f admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:31.030227 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b6fd4-3fa1-4be5-bc26-2a1b1e69924f admin admin] PATH_INFO: `/v3/projects/54bb4955000143f480eadeca634a8dd7/users/2783fecdbf6b475fb5ae16006df95dd1/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:31.032935 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:31.050212 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b6fd4-3fa1-4be5-bc26-2a1b1e69924f admin admin] RBAC: Authorizing `identity:create_grant(project_id=54bb4955000143f480eadeca634a8dd7, user_id=2783fecdbf6b475fb5ae16006df95dd1, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:31.053078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b6fd4-3fa1-4be5-bc26-2a1b1e69924f admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:31.069365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1282/2577] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:19:31 2022] PUT /identity/v3/projects/54bb4955000143f480eadeca634a8dd7/users/2783fecdbf6b475fb5ae16006df95dd1/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:31.079963 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8f305-1215-4b70-a7b6-5f8a1112ddf0 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:31.080180 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8f305-1215-4b70-a7b6-5f8a1112ddf0 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:31.080427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8f305-1215-4b70-a7b6-5f8a1112ddf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:31.113044 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-1ba8f305-1215-4b70-a7b6-5f8a1112ddf0 None None] MFA Rules not processed for user `2783fecdbf6b475fb5ae16006df95dd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:31.114347 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-1ba8f305-1215-4b70-a7b6-5f8a1112ddf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:31.186215 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1296/2578] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:19:31.198489 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b6fd4-3fa1-4be5-bc26-2a1b1e69924f admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:31.200409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-ed5b6fd4-3fa1-4be5-bc26-2a1b1e69924f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:31.296268 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b6fd4-3fa1-4be5-bc26-2a1b1e69924f admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:31.302438 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7898eb0-e214-4906-8d52-16b868362407 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:31.303044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7898eb0-e214-4906-8d52-16b868362407 service nova] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:31.303645 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7898eb0-e214-4906-8d52-16b868362407 service nova] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:31.303645 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7898eb0-e214-4906-8d52-16b868362407 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:31.312335 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7898eb0-e214-4906-8d52-16b868362407 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:31.314366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7898eb0-e214-4906-8d52-16b868362407 service nova] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:31.324427 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1283/2579] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:19:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:19:39.491776 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba8f305-1215-4b70-a7b6-5f8a1112ddf0 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:39.494988 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-1ba8f305-1215-4b70-a7b6-5f8a1112ddf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:39.576481 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba8f305-1215-4b70-a7b6-5f8a1112ddf0 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:39.580982 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa914e-db0d-4ccf-9ed4-be649be9b369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:39.581702 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa914e-db0d-4ccf-9ed4-be649be9b369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:39.581964 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa914e-db0d-4ccf-9ed4-be649be9b369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:39.582230 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa914e-db0d-4ccf-9ed4-be649be9b369 admin admin] PATH_INFO: `/v3/users/d1d86083b55244b8948cd1e007d3829e` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:39.583702 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaa914e-db0d-4ccf-9ed4-be649be9b369 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1d86083b55244b8948cd1e007d3829e)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:39.585711 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaa914e-db0d-4ccf-9ed4-be649be9b369 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:39.645071 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-bfaa914e-db0d-4ccf-9ed4-be649be9b369 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1d86083b55244b8948cd1e007d3829e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:39.653981 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-bfaa914e-db0d-4ccf-9ed4-be649be9b369 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1d86083b55244b8948cd1e007d3829e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:39.669108 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-bfaa914e-db0d-4ccf-9ed4-be649be9b369 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1d86083b55244b8948cd1e007d3829e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:39.669960 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-bfaa914e-db0d-4ccf-9ed4-be649be9b369 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1d86083b55244b8948cd1e007d3829e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:39.679377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:39.681845 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-bfaa914e-db0d-4ccf-9ed4-be649be9b369 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1d86083b55244b8948cd1e007d3829e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:39.692944 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1297/2580] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:19:39 2022] DELETE /identity/v3/users/d1d86083b55244b8948cd1e007d3829e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:39.699755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0d26e-72a7-4927-8a57-ce64e8cd5262 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:39.700363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0d26e-72a7-4927-8a57-ce64e8cd5262 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:39.700469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0d26e-72a7-4927-8a57-ce64e8cd5262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:39.725766 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-2fc0d26e-72a7-4927-8a57-ce64e8cd5262 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:39.726885 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-2fc0d26e-72a7-4927-8a57-ce64e8cd5262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:39.798982 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1284/2581] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:39.812386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa914e-db0d-4ccf-9ed4-be649be9b369 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:39.814817 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-bfaa914e-db0d-4ccf-9ed4-be649be9b369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:39.883100 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa914e-db0d-4ccf-9ed4-be649be9b369 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:39.892426 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d1be8-2ede-4b0f-afbb-56bb3d19ff48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:39.892426 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d1be8-2ede-4b0f-afbb-56bb3d19ff48 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:39.892426 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d1be8-2ede-4b0f-afbb-56bb3d19ff48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:39.892426 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d1be8-2ede-4b0f-afbb-56bb3d19ff48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:39.903564 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d1be8-2ede-4b0f-afbb-56bb3d19ff48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:39.904952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d1be8-2ede-4b0f-afbb-56bb3d19ff48 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:39.917115 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1298/2582] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:19:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:19:40.257080 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0d26e-72a7-4927-8a57-ce64e8cd5262 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:40.258973 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-2fc0d26e-72a7-4927-8a57-ce64e8cd5262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:40.272777 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0d26e-72a7-4927-8a57-ce64e8cd5262 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:40.278497 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc998e-fede-488a-9e36-69a3a3b8713b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:40.279300 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc998e-fede-488a-9e36-69a3a3b8713b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:40.279677 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc998e-fede-488a-9e36-69a3a3b8713b admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:40.280073 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc998e-fede-488a-9e36-69a3a3b8713b admin admin] PATH_INFO: `/v3/projects/0d2d462ebe4c477b9712e391bfb85d96` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:40.281730 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc998e-fede-488a-9e36-69a3a3b8713b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d2d462ebe4c477b9712e391bfb85d96)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:40.283726 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc998e-fede-488a-9e36-69a3a3b8713b admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:40.345112 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-16dc998e-fede-488a-9e36-69a3a3b8713b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d2d462ebe4c477b9712e391bfb85d96', 'request_id': 'req-16dc998e-fede-488a-9e36-69a3a3b8713b'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:40.354017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-16dc998e-fede-488a-9e36-69a3a3b8713b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d2d462ebe4c477b9712e391bfb85d96', 'request_id': 'req-16dc998e-fede-488a-9e36-69a3a3b8713b'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:40.360394 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:40.362123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-16dc998e-fede-488a-9e36-69a3a3b8713b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d2d462ebe4c477b9712e391bfb85d96', 'request_id': 'req-16dc998e-fede-488a-9e36-69a3a3b8713b'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:40.383658 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-16dc998e-fede-488a-9e36-69a3a3b8713b admin admin] The token cache is being invalidate because project 0d2d462ebe4c477b9712e391bfb85d96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:19:40.383938 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-16dc998e-fede-488a-9e36-69a3a3b8713b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:40.384675 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-16dc998e-fede-488a-9e36-69a3a3b8713b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:40.386348 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1285/2583] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:19:40 2022] DELETE /identity/v3/projects/0d2d462ebe4c477b9712e391bfb85d96 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:40.919250 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d1be8-2ede-4b0f-afbb-56bb3d19ff48 service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:40.919873 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-c64d1be8-2ede-4b0f-afbb-56bb3d19ff48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:40.933122 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d1be8-2ede-4b0f-afbb-56bb3d19ff48 service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:41.004767 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f084157-363d-41fc-9eab-135260d228f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:41.005479 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f084157-363d-41fc-9eab-135260d228f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:41.005753 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f084157-363d-41fc-9eab-135260d228f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:41.006020 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f084157-363d-41fc-9eab-135260d228f7 admin admin] PATH_INFO: `/v3/users/2783fecdbf6b475fb5ae16006df95dd1` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:41.007219 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f084157-363d-41fc-9eab-135260d228f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2783fecdbf6b475fb5ae16006df95dd1)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:41.009168 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f084157-363d-41fc-9eab-135260d228f7 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:41.065684 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5f084157-363d-41fc-9eab-135260d228f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2783fecdbf6b475fb5ae16006df95dd1'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:41.076252 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5f084157-363d-41fc-9eab-135260d228f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2783fecdbf6b475fb5ae16006df95dd1'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:41.092452 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5f084157-363d-41fc-9eab-135260d228f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2783fecdbf6b475fb5ae16006df95dd1'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:41.093233 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5f084157-363d-41fc-9eab-135260d228f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2783fecdbf6b475fb5ae16006df95dd1'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:41.108929 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5f084157-363d-41fc-9eab-135260d228f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2783fecdbf6b475fb5ae16006df95dd1'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:41.116162 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1299/2584] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:19:40 2022] DELETE /identity/v3/users/2783fecdbf6b475fb5ae16006df95dd1 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:41.121117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf53cf-a9d3-44a3-94f0-ce3c42078a45 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:41.121959 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf53cf-a9d3-44a3-94f0-ce3c42078a45 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:41.121959 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf53cf-a9d3-44a3-94f0-ce3c42078a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:41.143541 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-7eaf53cf-a9d3-44a3-94f0-ce3c42078a45 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:41.144783 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-7eaf53cf-a9d3-44a3-94f0-ce3c42078a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:41.211198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1286/2585] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:41.223110 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f084157-363d-41fc-9eab-135260d228f7 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:41.224805 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-5f084157-363d-41fc-9eab-135260d228f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:41.248505 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:41.301514 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f084157-363d-41fc-9eab-135260d228f7 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:41.305550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c88df5-6526-442e-8627-c40e0f30d6f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:41.306200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c88df5-6526-442e-8627-c40e0f30d6f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:41.306613 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c88df5-6526-442e-8627-c40e0f30d6f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:41.306893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c88df5-6526-442e-8627-c40e0f30d6f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:41.319268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c88df5-6526-442e-8627-c40e0f30d6f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:41.320978 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c88df5-6526-442e-8627-c40e0f30d6f9 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:41.333215 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1300/2586] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:19:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:19:41.863182 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf53cf-a9d3-44a3-94f0-ce3c42078a45 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:41.864740 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-7eaf53cf-a9d3-44a3-94f0-ce3c42078a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:41.879791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf53cf-a9d3-44a3-94f0-ce3c42078a45 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:41.884830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024ac40-17b2-4a56-ae56-4be108dd54cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:41.885525 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024ac40-17b2-4a56-ae56-4be108dd54cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:41.885803 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024ac40-17b2-4a56-ae56-4be108dd54cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:41.886163 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024ac40-17b2-4a56-ae56-4be108dd54cf admin admin] PATH_INFO: `/v3/projects/54bb4955000143f480eadeca634a8dd7` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:41.887354 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024ac40-17b2-4a56-ae56-4be108dd54cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=54bb4955000143f480eadeca634a8dd7)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:41.889351 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024ac40-17b2-4a56-ae56-4be108dd54cf admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:41.945257 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d024ac40-17b2-4a56-ae56-4be108dd54cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54bb4955000143f480eadeca634a8dd7', 'request_id': 'req-d024ac40-17b2-4a56-ae56-4be108dd54cf'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:41.954703 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d024ac40-17b2-4a56-ae56-4be108dd54cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54bb4955000143f480eadeca634a8dd7', 'request_id': 'req-d024ac40-17b2-4a56-ae56-4be108dd54cf'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:41.965707 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d024ac40-17b2-4a56-ae56-4be108dd54cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54bb4955000143f480eadeca634a8dd7', 'request_id': 'req-d024ac40-17b2-4a56-ae56-4be108dd54cf'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:41.990272 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d024ac40-17b2-4a56-ae56-4be108dd54cf admin admin] The token cache is being invalidate because project 54bb4955000143f480eadeca634a8dd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:19:41.990777 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d024ac40-17b2-4a56-ae56-4be108dd54cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:41.991878 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-d024ac40-17b2-4a56-ae56-4be108dd54cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:41.994540 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1287/2587] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:19:41 2022] DELETE /identity/v3/projects/54bb4955000143f480eadeca634a8dd7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:42.001041 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6b299-18bf-46ea-9cef-97a5472fce14 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:42.001687 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6b299-18bf-46ea-9cef-97a5472fce14 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:42.001892 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6b299-18bf-46ea-9cef-97a5472fce14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:42.036430 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-82c6b299-18bf-46ea-9cef-97a5472fce14 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:42.038615 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-82c6b299-18bf-46ea-9cef-97a5472fce14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:42.148903 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1301/2588] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:42.166958 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900974b9-93f1-415c-81d1-77be4cb25dfc None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:42.168005 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900974b9-93f1-415c-81d1-77be4cb25dfc None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:42.168005 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900974b9-93f1-415c-81d1-77be4cb25dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:42.203455 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-900974b9-93f1-415c-81d1-77be4cb25dfc None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:42.204704 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-900974b9-93f1-415c-81d1-77be4cb25dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:42.217630 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1288/2589] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:19:42.222550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6b299-18bf-46ea-9cef-97a5472fce14 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:42.234370 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6b299-18bf-46ea-9cef-97a5472fce14 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:42.240377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b34597-f4b0-42fe-b74f-6f082f681702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:42.241062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b34597-f4b0-42fe-b74f-6f082f681702 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:42.241366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b34597-f4b0-42fe-b74f-6f082f681702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:42.241635 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b34597-f4b0-42fe-b74f-6f082f681702 admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:42.242295 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b34597-f4b0-42fe-b74f-6f082f681702 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:19:42.242671 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b34597-f4b0-42fe-b74f-6f082f681702 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:42.244363 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b34597-f4b0-42fe-b74f-6f082f681702 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:42.263988 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1302/2590] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:19:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:19:42.269296 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900974b9-93f1-415c-81d1-77be4cb25dfc None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:42.279914 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900974b9-93f1-415c-81d1-77be4cb25dfc None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:42.284856 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c848d-4984-481a-bb4c-380422d50775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:42.285540 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c848d-4984-481a-bb4c-380422d50775 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:42.285812 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c848d-4984-481a-bb4c-380422d50775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:42.286079 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c848d-4984-481a-bb4c-380422d50775 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:42.286763 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c848d-4984-481a-bb4c-380422d50775 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:42.288826 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c848d-4984-481a-bb4c-380422d50775 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:42.321254 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1289/2591] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:19:42 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:19:42.326113 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b34597-f4b0-42fe-b74f-6f082f681702 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:42.337303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b34597-f4b0-42fe-b74f-6f082f681702 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:42.404408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c4d0f-c0b0-4ebb-a5a9-3b36fe951293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:42.404957 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c4d0f-c0b0-4ebb-a5a9-3b36fe951293 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:42.405304 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c4d0f-c0b0-4ebb-a5a9-3b36fe951293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:42.405621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c4d0f-c0b0-4ebb-a5a9-3b36fe951293 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:42.406365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6c4d0f-c0b0-4ebb-a5a9-3b36fe951293 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:42.408612 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6c4d0f-c0b0-4ebb-a5a9-3b36fe951293 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:42.434388 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1303/2592] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:19:42 2022] POST /identity/v3/users => generated 294 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:19:42.439962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c848d-4984-481a-bb4c-380422d50775 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:42.454624 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c848d-4984-481a-bb4c-380422d50775 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:42.459639 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760e84c-5ee9-45ea-b353-7cab4ca3e8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:42.460245 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760e84c-5ee9-45ea-b353-7cab4ca3e8d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:42.460522 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760e84c-5ee9-45ea-b353-7cab4ca3e8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:42.460786 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760e84c-5ee9-45ea-b353-7cab4ca3e8d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:42.461478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760e84c-5ee9-45ea-b353-7cab4ca3e8d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:42.463116 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760e84c-5ee9-45ea-b353-7cab4ca3e8d5 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:42.476065 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1290/2593] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:19:42.480986 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c4d0f-c0b0-4ebb-a5a9-3b36fe951293 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:42.491812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c4d0f-c0b0-4ebb-a5a9-3b36fe951293 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:42.497004 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807883ad-75ce-4bfe-8b6c-e8dbb5d49f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:42.497683 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807883ad-75ce-4bfe-8b6c-e8dbb5d49f66 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:42.497958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807883ad-75ce-4bfe-8b6c-e8dbb5d49f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:42.498224 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807883ad-75ce-4bfe-8b6c-e8dbb5d49f66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:42.498885 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807883ad-75ce-4bfe-8b6c-e8dbb5d49f66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:42.500492 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807883ad-75ce-4bfe-8b6c-e8dbb5d49f66 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:42.512774 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1304/2594] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:19:42.517210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760e84c-5ee9-45ea-b353-7cab4ca3e8d5 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:42.528107 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760e84c-5ee9-45ea-b353-7cab4ca3e8d5 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:42.533785 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c33066-a0fa-4c82-9bb6-f4cde1eb8de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:42.534444 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c33066-a0fa-4c82-9bb6-f4cde1eb8de0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:42.534716 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c33066-a0fa-4c82-9bb6-f4cde1eb8de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:42.534982 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c33066-a0fa-4c82-9bb6-f4cde1eb8de0 admin admin] PATH_INFO: `/v3/projects/188c6c50f0b64efdb759c6487fbdf347/users/b3e91511c015437c8fd0736663eda190/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:42.552212 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c33066-a0fa-4c82-9bb6-f4cde1eb8de0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=188c6c50f0b64efdb759c6487fbdf347, user_id=b3e91511c015437c8fd0736663eda190, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:42.554941 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c33066-a0fa-4c82-9bb6-f4cde1eb8de0 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:42.572126 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1291/2595] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:19:42 2022] PUT /identity/v3/projects/188c6c50f0b64efdb759c6487fbdf347/users/b3e91511c015437c8fd0736663eda190/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:42.583253 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09fdf79-1f40-4371-bc25-b09b9d56af95 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:42.583509 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09fdf79-1f40-4371-bc25-b09b9d56af95 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:42.583769 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09fdf79-1f40-4371-bc25-b09b9d56af95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:42.614738 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-c09fdf79-1f40-4371-bc25-b09b9d56af95 None None] MFA Rules not processed for user `b3e91511c015437c8fd0736663eda190`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:42.616194 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-c09fdf79-1f40-4371-bc25-b09b9d56af95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:42.630733 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:42.692707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1305/2596] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:42.697606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c33066-a0fa-4c82-9bb6-f4cde1eb8de0 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:42.711137 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:42.768732 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c33066-a0fa-4c82-9bb6-f4cde1eb8de0 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:42.773943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164dcd3-58a8-459b-8a2c-b3f6b656b8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:42.774561 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164dcd3-58a8-459b-8a2c-b3f6b656b8a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:42.774837 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164dcd3-58a8-459b-8a2c-b3f6b656b8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:42.775103 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164dcd3-58a8-459b-8a2c-b3f6b656b8a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:42.775743 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a164dcd3-58a8-459b-8a2c-b3f6b656b8a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:42.777767 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a164dcd3-58a8-459b-8a2c-b3f6b656b8a0 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:42.809237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1292/2597] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:19:42 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:19:42.814080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09fdf79-1f40-4371-bc25-b09b9d56af95 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:42.826046 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09fdf79-1f40-4371-bc25-b09b9d56af95 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:42.890359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab58051-9606-4f0c-9ef6-eae87bcdcdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:42.890985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab58051-9606-4f0c-9ef6-eae87bcdcdaa admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:42.891256 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab58051-9606-4f0c-9ef6-eae87bcdcdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:42.891519 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab58051-9606-4f0c-9ef6-eae87bcdcdaa admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:42.892170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab58051-9606-4f0c-9ef6-eae87bcdcdaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:42.894164 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab58051-9606-4f0c-9ef6-eae87bcdcdaa admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:42.921927 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1306/2598] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:19:42 2022] POST /identity/v3/users => generated 299 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:19:42.926270 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164dcd3-58a8-459b-8a2c-b3f6b656b8a0 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:42.938969 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164dcd3-58a8-459b-8a2c-b3f6b656b8a0 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:42.944934 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae6a6c-6d49-4b82-a4f4-4c8ebede203f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:42.945515 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae6a6c-6d49-4b82-a4f4-4c8ebede203f admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:42.945795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae6a6c-6d49-4b82-a4f4-4c8ebede203f admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:42.946056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae6a6c-6d49-4b82-a4f4-4c8ebede203f admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:42.946720 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae6a6c-6d49-4b82-a4f4-4c8ebede203f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:42.948370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae6a6c-6d49-4b82-a4f4-4c8ebede203f admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:42.960792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1293/2599] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:19:42.965562 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab58051-9606-4f0c-9ef6-eae87bcdcdaa admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:42.976917 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab58051-9606-4f0c-9ef6-eae87bcdcdaa admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:42.984035 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef843d9-f939-4688-a8ee-461df53d5147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:42.984741 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef843d9-f939-4688-a8ee-461df53d5147 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:42.985007 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef843d9-f939-4688-a8ee-461df53d5147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:42.985320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef843d9-f939-4688-a8ee-461df53d5147 admin admin] PATH_INFO: `/v3/projects/f194ef7f7f2748e387b3d2582e8f5b5b/users/eba88e20a5814e719fc20954547b8007/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:43.003513 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef843d9-f939-4688-a8ee-461df53d5147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f194ef7f7f2748e387b3d2582e8f5b5b, user_id=eba88e20a5814e719fc20954547b8007, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:43.006265 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef843d9-f939-4688-a8ee-461df53d5147 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:43.025759 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1307/2600] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:19:42 2022] PUT /identity/v3/projects/f194ef7f7f2748e387b3d2582e8f5b5b/users/eba88e20a5814e719fc20954547b8007/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:43.031224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae6a6c-6d49-4b82-a4f4-4c8ebede203f admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:43.041820 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae6a6c-6d49-4b82-a4f4-4c8ebede203f admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:43.106140 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c3569-a3c6-4f03-b17c-36ad0ac57845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:43.106644 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c3569-a3c6-4f03-b17c-36ad0ac57845 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:43.106992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c3569-a3c6-4f03-b17c-36ad0ac57845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:43.107301 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c3569-a3c6-4f03-b17c-36ad0ac57845 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:43.108061 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28c3569-a3c6-4f03-b17c-36ad0ac57845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:43.109757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28c3569-a3c6-4f03-b17c-36ad0ac57845 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:43.121084 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1294/2601] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:19:43.126702 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef843d9-f939-4688-a8ee-461df53d5147 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:43.138033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef843d9-f939-4688-a8ee-461df53d5147 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:43.143286 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a40e2c-7bf7-4a01-b01d-1fd9d561c7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:43.144027 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a40e2c-7bf7-4a01-b01d-1fd9d561c7e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:43.144350 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a40e2c-7bf7-4a01-b01d-1fd9d561c7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:43.144618 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a40e2c-7bf7-4a01-b01d-1fd9d561c7e7 admin admin] PATH_INFO: `/v3/projects/f194ef7f7f2748e387b3d2582e8f5b5b/users/eba88e20a5814e719fc20954547b8007/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:43.146888 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a40e2c-7bf7-4a01-b01d-1fd9d561c7e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f194ef7f7f2748e387b3d2582e8f5b5b, user_id=eba88e20a5814e719fc20954547b8007, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:43.149765 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a40e2c-7bf7-4a01-b01d-1fd9d561c7e7 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:43.161710 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1308/2602] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:19:43 2022] PUT /identity/v3/projects/f194ef7f7f2748e387b3d2582e8f5b5b/users/eba88e20a5814e719fc20954547b8007/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:43.173585 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf47af-b452-436d-b359-b9eb9a4be76b None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:43.174171 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf47af-b452-436d-b359-b9eb9a4be76b None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:43.174527 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf47af-b452-436d-b359-b9eb9a4be76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:43.208213 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-a1bf47af-b452-436d-b359-b9eb9a4be76b None None] MFA Rules not processed for user `eba88e20a5814e719fc20954547b8007`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:43.209392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-a1bf47af-b452-436d-b359-b9eb9a4be76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:43.290223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1295/2603] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:43.302080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a40e2c-7bf7-4a01-b01d-1fd9d561c7e7 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:43.303995 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-e5a40e2c-7bf7-4a01-b01d-1fd9d561c7e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:43.369696 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a40e2c-7bf7-4a01-b01d-1fd9d561c7e7 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:43.373614 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b5eef-4f4b-4910-88a6-832a38509e3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:43.374240 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b5eef-4f4b-4910-88a6-832a38509e3e service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:43.374514 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b5eef-4f4b-4910-88a6-832a38509e3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:43.374784 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b5eef-4f4b-4910-88a6-832a38509e3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:43.384703 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b5eef-4f4b-4910-88a6-832a38509e3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:43.386221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b5eef-4f4b-4910-88a6-832a38509e3e service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:43.396616 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1309/2604] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:19:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:19:43.471053 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf47af-b452-436d-b359-b9eb9a4be76b None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:43.481154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:43.551943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf47af-b452-436d-b359-b9eb9a4be76b None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:43.556897 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b548a0-c694-4f14-96d6-c90b40d8d82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:43.557577 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b548a0-c694-4f14-96d6-c90b40d8d82e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:43.557854 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b548a0-c694-4f14-96d6-c90b40d8d82e admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:43.558124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b548a0-c694-4f14-96d6-c90b40d8d82e admin admin] PATH_INFO: `/v3/users/b3e91511c015437c8fd0736663eda190` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:43.559330 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b548a0-c694-4f14-96d6-c90b40d8d82e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3e91511c015437c8fd0736663eda190)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:43.561315 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b548a0-c694-4f14-96d6-c90b40d8d82e admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:43.613626 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-34b548a0-c694-4f14-96d6-c90b40d8d82e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3e91511c015437c8fd0736663eda190'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:43.622643 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-34b548a0-c694-4f14-96d6-c90b40d8d82e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3e91511c015437c8fd0736663eda190'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:43.637316 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-34b548a0-c694-4f14-96d6-c90b40d8d82e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3e91511c015437c8fd0736663eda190'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:43.638465 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-34b548a0-c694-4f14-96d6-c90b40d8d82e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3e91511c015437c8fd0736663eda190'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:43.646667 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-34b548a0-c694-4f14-96d6-c90b40d8d82e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3e91511c015437c8fd0736663eda190'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:43.654270 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1296/2605] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:19:43 2022] DELETE /identity/v3/users/b3e91511c015437c8fd0736663eda190 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:43.659304 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446401e-e1ba-418d-9319-5d70a0ef4953 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:43.659602 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446401e-e1ba-418d-9319-5d70a0ef4953 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:43.659929 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446401e-e1ba-418d-9319-5d70a0ef4953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:43.680672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-f446401e-e1ba-418d-9319-5d70a0ef4953 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:43.681944 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-f446401e-e1ba-418d-9319-5d70a0ef4953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:43.686176 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:43.758396 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1310/2606] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:19:43.768625 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b548a0-c694-4f14-96d6-c90b40d8d82e admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:43.770150 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-34b548a0-c694-4f14-96d6-c90b40d8d82e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:43.833969 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b548a0-c694-4f14-96d6-c90b40d8d82e admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:43.837944 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab26e5-a46d-46f7-a00f-773bcb353197 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:43.838587 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab26e5-a46d-46f7-a00f-773bcb353197 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:43.838844 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab26e5-a46d-46f7-a00f-773bcb353197 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:43.839171 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab26e5-a46d-46f7-a00f-773bcb353197 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:43.849247 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab26e5-a46d-46f7-a00f-773bcb353197 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:43.850709 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab26e5-a46d-46f7-a00f-773bcb353197 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:43.861055 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1297/2607] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:19:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:19:44.165087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446401e-e1ba-418d-9319-5d70a0ef4953 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:44.167051 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-f446401e-e1ba-418d-9319-5d70a0ef4953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:44.184707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446401e-e1ba-418d-9319-5d70a0ef4953 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:44.194868 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ce3e1-9837-465f-9303-9a99cbfcbe67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:44.197162 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ce3e1-9837-465f-9303-9a99cbfcbe67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:44.197687 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ce3e1-9837-465f-9303-9a99cbfcbe67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:44.198137 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ce3e1-9837-465f-9303-9a99cbfcbe67 admin admin] PATH_INFO: `/v3/projects/188c6c50f0b64efdb759c6487fbdf347` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:44.200133 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ce3e1-9837-465f-9303-9a99cbfcbe67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=188c6c50f0b64efdb759c6487fbdf347)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:44.203065 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ce3e1-9837-465f-9303-9a99cbfcbe67 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:44.260097 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-3a0ce3e1-9837-465f-9303-9a99cbfcbe67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '188c6c50f0b64efdb759c6487fbdf347', 'request_id': 'req-3a0ce3e1-9837-465f-9303-9a99cbfcbe67'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:44.266339 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-3a0ce3e1-9837-465f-9303-9a99cbfcbe67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '188c6c50f0b64efdb759c6487fbdf347', 'request_id': 'req-3a0ce3e1-9837-465f-9303-9a99cbfcbe67'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:44.272934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-3a0ce3e1-9837-465f-9303-9a99cbfcbe67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '188c6c50f0b64efdb759c6487fbdf347', 'request_id': 'req-3a0ce3e1-9837-465f-9303-9a99cbfcbe67'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:44.287171 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-3a0ce3e1-9837-465f-9303-9a99cbfcbe67 admin admin] The token cache is being invalidate because project 188c6c50f0b64efdb759c6487fbdf347 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:19:44.287416 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-3a0ce3e1-9837-465f-9303-9a99cbfcbe67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:44.288185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-3a0ce3e1-9837-465f-9303-9a99cbfcbe67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:44.289766 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1311/2608] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:19:44 2022] DELETE /identity/v3/projects/188c6c50f0b64efdb759c6487fbdf347 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:44.294025 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab26e5-a46d-46f7-a00f-773bcb353197 service neutron] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:44.303903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab26e5-a46d-46f7-a00f-773bcb353197 service neutron] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:44.331111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:44.364572 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a005747-4303-4e43-aebd-fe8d69430c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:44.365292 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a005747-4303-4e43-aebd-fe8d69430c85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:44.365565 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a005747-4303-4e43-aebd-fe8d69430c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:44.365828 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a005747-4303-4e43-aebd-fe8d69430c85 admin admin] PATH_INFO: `/v3/users/eba88e20a5814e719fc20954547b8007` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:44.366963 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a005747-4303-4e43-aebd-fe8d69430c85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eba88e20a5814e719fc20954547b8007)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:44.368870 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a005747-4303-4e43-aebd-fe8d69430c85 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:44.433177 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-2a005747-4303-4e43-aebd-fe8d69430c85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eba88e20a5814e719fc20954547b8007'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:44.434657 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-2a005747-4303-4e43-aebd-fe8d69430c85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eba88e20a5814e719fc20954547b8007'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:44.448030 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-2a005747-4303-4e43-aebd-fe8d69430c85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eba88e20a5814e719fc20954547b8007'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:44.448870 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-2a005747-4303-4e43-aebd-fe8d69430c85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eba88e20a5814e719fc20954547b8007'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:44.455938 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-2a005747-4303-4e43-aebd-fe8d69430c85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eba88e20a5814e719fc20954547b8007'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:44.462084 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1298/2609] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:19:44 2022] DELETE /identity/v3/users/eba88e20a5814e719fc20954547b8007 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:44.781899 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ce3e1-9837-465f-9303-9a99cbfcbe67 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:44.783987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-3a0ce3e1-9837-465f-9303-9a99cbfcbe67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:44.806376 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ce3e1-9837-465f-9303-9a99cbfcbe67 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:44.887397 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480bd13-de66-49cc-9203-a973c79f1aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:44.888041 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480bd13-de66-49cc-9203-a973c79f1aca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:44.888321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480bd13-de66-49cc-9203-a973c79f1aca admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:44.888587 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480bd13-de66-49cc-9203-a973c79f1aca admin admin] PATH_INFO: `/v3/projects/f194ef7f7f2748e387b3d2582e8f5b5b` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:44.889845 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5480bd13-de66-49cc-9203-a973c79f1aca admin admin] RBAC: Authorizing `identity:delete_project(project_id=f194ef7f7f2748e387b3d2582e8f5b5b)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:44.891787 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5480bd13-de66-49cc-9203-a973c79f1aca admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:44.938648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:44.951167 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5480bd13-de66-49cc-9203-a973c79f1aca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f194ef7f7f2748e387b3d2582e8f5b5b', 'request_id': 'req-5480bd13-de66-49cc-9203-a973c79f1aca'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:44.957500 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5480bd13-de66-49cc-9203-a973c79f1aca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f194ef7f7f2748e387b3d2582e8f5b5b', 'request_id': 'req-5480bd13-de66-49cc-9203-a973c79f1aca'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:44.964067 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5480bd13-de66-49cc-9203-a973c79f1aca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f194ef7f7f2748e387b3d2582e8f5b5b', 'request_id': 'req-5480bd13-de66-49cc-9203-a973c79f1aca'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:44.980023 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5480bd13-de66-49cc-9203-a973c79f1aca admin admin] The token cache is being invalidate because project f194ef7f7f2748e387b3d2582e8f5b5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:19:44.980333 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5480bd13-de66-49cc-9203-a973c79f1aca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:44.981258 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5480bd13-de66-49cc-9203-a973c79f1aca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:44.983338 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1312/2610] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:19:44 2022] DELETE /identity/v3/projects/f194ef7f7f2748e387b3d2582e8f5b5b => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:44.988467 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1731c-c14d-45bb-a3cd-6819c53a29fe None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:44.988747 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1731c-c14d-45bb-a3cd-6819c53a29fe None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:44.989009 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1731c-c14d-45bb-a3cd-6819c53a29fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:45.010189 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-21e1731c-c14d-45bb-a3cd-6819c53a29fe None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:45.011393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-21e1731c-c14d-45bb-a3cd-6819c53a29fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:45.078010 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1299/2611] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:45.088075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cdcef8-ce38-4994-80a2-41b4958e3904 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:45.088568 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cdcef8-ce38-4994-80a2-41b4958e3904 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:45.089056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cdcef8-ce38-4994-80a2-41b4958e3904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:45.118108 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-f0cdcef8-ce38-4994-80a2-41b4958e3904 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:45.119366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-f0cdcef8-ce38-4994-80a2-41b4958e3904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:45.128371 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1313/2612] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:45.133063 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1731c-c14d-45bb-a3cd-6819c53a29fe None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:45.144227 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1731c-c14d-45bb-a3cd-6819c53a29fe None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:45.149378 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd271d70-d75d-45d3-b3c2-d2bad1606b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:45.150001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd271d70-d75d-45d3-b3c2-d2bad1606b21 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:45.150276 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd271d70-d75d-45d3-b3c2-d2bad1606b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:45.150563 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd271d70-d75d-45d3-b3c2-d2bad1606b21 admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:45.151203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd271d70-d75d-45d3-b3c2-d2bad1606b21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:19:45.151505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd271d70-d75d-45d3-b3c2-d2bad1606b21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:45.153151 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd271d70-d75d-45d3-b3c2-d2bad1606b21 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:45.166720 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1300/2613] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:19:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:19:45.172015 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cdcef8-ce38-4994-80a2-41b4958e3904 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:45.184962 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cdcef8-ce38-4994-80a2-41b4958e3904 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:45.193669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e234e-431a-477c-9696-3c6f6b8b9ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:45.194516 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e234e-431a-477c-9696-3c6f6b8b9ccf admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:45.194867 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e234e-431a-477c-9696-3c6f6b8b9ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:45.195197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e234e-431a-477c-9696-3c6f6b8b9ccf admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:45.196107 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e234e-431a-477c-9696-3c6f6b8b9ccf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:45.198701 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e234e-431a-477c-9696-3c6f6b8b9ccf admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:45.234773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1314/2614] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:19:45 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:19:45.241018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd271d70-d75d-45d3-b3c2-d2bad1606b21 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:45.252631 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd271d70-d75d-45d3-b3c2-d2bad1606b21 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:45.289053 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:45.319091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5af2d3-3985-4fdc-89b9-d3785c9f3750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:45.319811 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5af2d3-3985-4fdc-89b9-d3785c9f3750 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:45.319923 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5af2d3-3985-4fdc-89b9-d3785c9f3750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:45.320214 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5af2d3-3985-4fdc-89b9-d3785c9f3750 admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:45.320845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5af2d3-3985-4fdc-89b9-d3785c9f3750 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:45.322792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5af2d3-3985-4fdc-89b9-d3785c9f3750 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:45.346483 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1301/2615] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:19:45 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:19:45.352445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e234e-431a-477c-9696-3c6f6b8b9ccf admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:45.365534 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e234e-431a-477c-9696-3c6f6b8b9ccf admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:45.372887 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3271a7-2b15-426f-98c7-1b35b364f0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:45.373526 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3271a7-2b15-426f-98c7-1b35b364f0df admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:45.373796 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3271a7-2b15-426f-98c7-1b35b364f0df admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:45.374059 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3271a7-2b15-426f-98c7-1b35b364f0df admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:45.374706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3271a7-2b15-426f-98c7-1b35b364f0df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:45.376308 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3271a7-2b15-426f-98c7-1b35b364f0df admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:45.389105 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1315/2616] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:19:45.394212 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5af2d3-3985-4fdc-89b9-d3785c9f3750 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:45.405468 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5af2d3-3985-4fdc-89b9-d3785c9f3750 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:45.411999 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78616f4-d351-4325-833e-bd77fa770aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:45.412939 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78616f4-d351-4325-833e-bd77fa770aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:45.413307 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78616f4-d351-4325-833e-bd77fa770aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:45.413610 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78616f4-d351-4325-833e-bd77fa770aa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:45.414399 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78616f4-d351-4325-833e-bd77fa770aa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:45.416074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78616f4-d351-4325-833e-bd77fa770aa6 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:45.427526 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1302/2617] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:19:45.432252 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3271a7-2b15-426f-98c7-1b35b364f0df admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:45.443344 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3271a7-2b15-426f-98c7-1b35b364f0df admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:45.448401 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a874959-8f47-4e20-af75-11c254ba3b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:45.449057 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a874959-8f47-4e20-af75-11c254ba3b15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:45.449383 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a874959-8f47-4e20-af75-11c254ba3b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:45.449653 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a874959-8f47-4e20-af75-11c254ba3b15 admin admin] PATH_INFO: `/v3/projects/47ec7f58837a47ce8bdb47afc4f4e836/users/946b63a5772844c580c0138a6ab2c6cd/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:45.465276 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a874959-8f47-4e20-af75-11c254ba3b15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47ec7f58837a47ce8bdb47afc4f4e836, user_id=946b63a5772844c580c0138a6ab2c6cd, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:45.467899 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a874959-8f47-4e20-af75-11c254ba3b15 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:45.483271 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1316/2618] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:19:45 2022] PUT /identity/v3/projects/47ec7f58837a47ce8bdb47afc4f4e836/users/946b63a5772844c580c0138a6ab2c6cd/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:45.493922 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a732135-6d30-4edf-82ce-8c194de1c990 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:45.494187 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a732135-6d30-4edf-82ce-8c194de1c990 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:45.494477 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a732135-6d30-4edf-82ce-8c194de1c990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:45.524061 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-7a732135-6d30-4edf-82ce-8c194de1c990 None None] MFA Rules not processed for user `946b63a5772844c580c0138a6ab2c6cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:45.525304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-7a732135-6d30-4edf-82ce-8c194de1c990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:45.597261 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1303/2619] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:45.609068 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a874959-8f47-4e20-af75-11c254ba3b15 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:45.611189 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-4a874959-8f47-4e20-af75-11c254ba3b15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:45.672001 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a874959-8f47-4e20-af75-11c254ba3b15 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:45.675894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3348b-f0b6-4ff6-9fda-e17b995aa777 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:45.676510 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3348b-f0b6-4ff6-9fda-e17b995aa777 service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:45.676775 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3348b-f0b6-4ff6-9fda-e17b995aa777 service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:45.677041 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3348b-f0b6-4ff6-9fda-e17b995aa777 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:45.686271 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a3348b-f0b6-4ff6-9fda-e17b995aa777 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:45.687697 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a3348b-f0b6-4ff6-9fda-e17b995aa777 service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:45.697531 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1317/2620] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:19:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:19:45.718730 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a732135-6d30-4edf-82ce-8c194de1c990 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:45.784624 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a732135-6d30-4edf-82ce-8c194de1c990 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:45.790688 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f883b0cd-0a26-4c77-ae83-6f1c545be73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:45.791470 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f883b0cd-0a26-4c77-ae83-6f1c545be73b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:45.791857 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f883b0cd-0a26-4c77-ae83-6f1c545be73b admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:45.792243 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f883b0cd-0a26-4c77-ae83-6f1c545be73b admin admin] PATH_INFO: `/v3/users/946b63a5772844c580c0138a6ab2c6cd` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:45.793421 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f883b0cd-0a26-4c77-ae83-6f1c545be73b admin admin] RBAC: Authorizing `identity:delete_user(user_id=946b63a5772844c580c0138a6ab2c6cd)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:45.795365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f883b0cd-0a26-4c77-ae83-6f1c545be73b admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:45.846254 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:45.848021 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-f883b0cd-0a26-4c77-ae83-6f1c545be73b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '946b63a5772844c580c0138a6ab2c6cd'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:45.856562 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-f883b0cd-0a26-4c77-ae83-6f1c545be73b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '946b63a5772844c580c0138a6ab2c6cd'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:45.869047 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-f883b0cd-0a26-4c77-ae83-6f1c545be73b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '946b63a5772844c580c0138a6ab2c6cd'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:45.869741 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-f883b0cd-0a26-4c77-ae83-6f1c545be73b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '946b63a5772844c580c0138a6ab2c6cd'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:45.876390 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-f883b0cd-0a26-4c77-ae83-6f1c545be73b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '946b63a5772844c580c0138a6ab2c6cd'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:45.882282 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1304/2621] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:19:45 2022] DELETE /identity/v3/users/946b63a5772844c580c0138a6ab2c6cd => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:45.886943 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106d6ea-332d-40c7-a2cc-bd5907ad6b18 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:45.887247 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106d6ea-332d-40c7-a2cc-bd5907ad6b18 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:45.887517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106d6ea-332d-40c7-a2cc-bd5907ad6b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:45.908394 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-e106d6ea-332d-40c7-a2cc-bd5907ad6b18 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:45.909720 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-e106d6ea-332d-40c7-a2cc-bd5907ad6b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:45.972651 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1318/2622] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:45.982704 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f883b0cd-0a26-4c77-ae83-6f1c545be73b admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:45.984187 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f883b0cd-0a26-4c77-ae83-6f1c545be73b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:46.044031 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f883b0cd-0a26-4c77-ae83-6f1c545be73b admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:46.047860 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d424eaf-b5a8-4d50-a37c-627a824281b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:46.048481 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d424eaf-b5a8-4d50-a37c-627a824281b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:46.048753 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d424eaf-b5a8-4d50-a37c-627a824281b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:46.049019 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d424eaf-b5a8-4d50-a37c-627a824281b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:46.058761 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d424eaf-b5a8-4d50-a37c-627a824281b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:46.060218 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d424eaf-b5a8-4d50-a37c-627a824281b2 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:46.070602 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1305/2623] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:19:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:19:46.381216 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e106d6ea-332d-40c7-a2cc-bd5907ad6b18 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:46.382888 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-e106d6ea-332d-40c7-a2cc-bd5907ad6b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:46.390130 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:46.397337 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e106d6ea-332d-40c7-a2cc-bd5907ad6b18 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:46.407738 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95a981-50d3-43d9-8742-c8cf69c39f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:46.408146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95a981-50d3-43d9-8742-c8cf69c39f8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:46.408438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95a981-50d3-43d9-8742-c8cf69c39f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:46.408727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95a981-50d3-43d9-8742-c8cf69c39f8e admin admin] PATH_INFO: `/v3/projects/47ec7f58837a47ce8bdb47afc4f4e836` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:46.410303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95a981-50d3-43d9-8742-c8cf69c39f8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=47ec7f58837a47ce8bdb47afc4f4e836)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:46.412537 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95a981-50d3-43d9-8742-c8cf69c39f8e admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:46.466728 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2a95a981-50d3-43d9-8742-c8cf69c39f8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47ec7f58837a47ce8bdb47afc4f4e836', 'request_id': 'req-2a95a981-50d3-43d9-8742-c8cf69c39f8e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:46.473860 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2a95a981-50d3-43d9-8742-c8cf69c39f8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47ec7f58837a47ce8bdb47afc4f4e836', 'request_id': 'req-2a95a981-50d3-43d9-8742-c8cf69c39f8e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:46.482130 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2a95a981-50d3-43d9-8742-c8cf69c39f8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47ec7f58837a47ce8bdb47afc4f4e836', 'request_id': 'req-2a95a981-50d3-43d9-8742-c8cf69c39f8e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:46.498232 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2a95a981-50d3-43d9-8742-c8cf69c39f8e admin admin] The token cache is being invalidate because project 47ec7f58837a47ce8bdb47afc4f4e836 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:19:46.498513 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2a95a981-50d3-43d9-8742-c8cf69c39f8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:46.499156 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2a95a981-50d3-43d9-8742-c8cf69c39f8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:46.500684 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1319/2624] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:19:46 2022] DELETE /identity/v3/projects/47ec7f58837a47ce8bdb47afc4f4e836 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:46.505660 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6daf65a-f390-4c72-8d62-953b1f6a8775 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:46.506150 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6daf65a-f390-4c72-8d62-953b1f6a8775 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:46.506244 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6daf65a-f390-4c72-8d62-953b1f6a8775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:46.526603 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-c6daf65a-f390-4c72-8d62-953b1f6a8775 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:46.527888 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c6daf65a-f390-4c72-8d62-953b1f6a8775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:46.610390 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1306/2625] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:46.618718 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc28fd-4cdf-4c26-affe-a2e316eaff7c None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:46.619022 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc28fd-4cdf-4c26-affe-a2e316eaff7c None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:46.619287 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc28fd-4cdf-4c26-affe-a2e316eaff7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:46.648397 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-fcdc28fd-4cdf-4c26-affe-a2e316eaff7c None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:46.650058 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-fcdc28fd-4cdf-4c26-affe-a2e316eaff7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:46.663467 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1320/2626] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:46.668270 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6daf65a-f390-4c72-8d62-953b1f6a8775 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:46.681941 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6daf65a-f390-4c72-8d62-953b1f6a8775 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:46.687901 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6706f6-138f-4bad-bcf2-79346262c70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:46.688929 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6706f6-138f-4bad-bcf2-79346262c70b admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:46.689233 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6706f6-138f-4bad-bcf2-79346262c70b admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:46.689495 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6706f6-138f-4bad-bcf2-79346262c70b admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:46.690144 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6706f6-138f-4bad-bcf2-79346262c70b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:19:46.690445 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6706f6-138f-4bad-bcf2-79346262c70b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:46.692047 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6706f6-138f-4bad-bcf2-79346262c70b admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:46.704636 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1307/2627] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:19:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:19:46.709433 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc28fd-4cdf-4c26-affe-a2e316eaff7c None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:46.719058 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc28fd-4cdf-4c26-affe-a2e316eaff7c None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:46.724640 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe8366-3833-4d91-8a05-802a49df76bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:46.725287 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe8366-3833-4d91-8a05-802a49df76bc admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:46.725563 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe8366-3833-4d91-8a05-802a49df76bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:46.725831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe8366-3833-4d91-8a05-802a49df76bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:46.726459 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe8366-3833-4d91-8a05-802a49df76bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:46.728407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe8366-3833-4d91-8a05-802a49df76bc admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:46.759506 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1321/2628] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:19:46 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:19:46.763952 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6706f6-138f-4bad-bcf2-79346262c70b admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:46.775212 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6706f6-138f-4bad-bcf2-79346262c70b admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:46.832682 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:46.847088 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962b70a-d7f4-45c8-b4d2-107390dcaebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:46.847904 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962b70a-d7f4-45c8-b4d2-107390dcaebe admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:46.848389 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962b70a-d7f4-45c8-b4d2-107390dcaebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:46.848714 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962b70a-d7f4-45c8-b4d2-107390dcaebe admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:46.849538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962b70a-d7f4-45c8-b4d2-107390dcaebe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:46.851859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962b70a-d7f4-45c8-b4d2-107390dcaebe admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:46.880969 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1308/2629] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:19:46 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:19:46.882468 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe8366-3833-4d91-8a05-802a49df76bc admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:46.893436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe8366-3833-4d91-8a05-802a49df76bc admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:46.898361 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c1b8c-aa7a-4f9b-b823-24acadc732f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:46.899048 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c1b8c-aa7a-4f9b-b823-24acadc732f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:46.899258 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c1b8c-aa7a-4f9b-b823-24acadc732f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:46.899520 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c1b8c-aa7a-4f9b-b823-24acadc732f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:46.900181 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c1b8c-aa7a-4f9b-b823-24acadc732f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:46.901865 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c1b8c-aa7a-4f9b-b823-24acadc732f2 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:46.912448 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1322/2630] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:19:46.917107 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962b70a-d7f4-45c8-b4d2-107390dcaebe admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:46.928511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962b70a-d7f4-45c8-b4d2-107390dcaebe admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:46.933576 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ebecef-1f4a-4832-9638-292b0092f6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:46.934185 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ebecef-1f4a-4832-9638-292b0092f6b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:46.934458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ebecef-1f4a-4832-9638-292b0092f6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:46.934749 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ebecef-1f4a-4832-9638-292b0092f6b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:46.935395 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ebecef-1f4a-4832-9638-292b0092f6b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:46.937001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ebecef-1f4a-4832-9638-292b0092f6b0 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:46.948780 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1309/2631] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:19:46.953856 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c1b8c-aa7a-4f9b-b823-24acadc732f2 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:46.965044 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c1b8c-aa7a-4f9b-b823-24acadc732f2 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:46.970471 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3678ca7-084f-414d-ab5b-c9a07d89f02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:46.971167 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3678ca7-084f-414d-ab5b-c9a07d89f02a admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:46.971452 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3678ca7-084f-414d-ab5b-c9a07d89f02a admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:46.971728 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3678ca7-084f-414d-ab5b-c9a07d89f02a admin admin] PATH_INFO: `/v3/projects/763fd6e7f5604e229987f55abf4130a0/users/f69cee9054584c978d8cce8baf004bc3/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:46.989464 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3678ca7-084f-414d-ab5b-c9a07d89f02a admin admin] RBAC: Authorizing `identity:create_grant(project_id=763fd6e7f5604e229987f55abf4130a0, user_id=f69cee9054584c978d8cce8baf004bc3, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:46.992186 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3678ca7-084f-414d-ab5b-c9a07d89f02a admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:47.007750 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1323/2632] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:19:46 2022] PUT /identity/v3/projects/763fd6e7f5604e229987f55abf4130a0/users/f69cee9054584c978d8cce8baf004bc3/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:19:47.019740 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd58b59-e242-4948-95af-390e98967fa2 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:47.020023 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd58b59-e242-4948-95af-390e98967fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:47.020294 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd58b59-e242-4948-95af-390e98967fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:47.052237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-6bd58b59-e242-4948-95af-390e98967fa2 None None] MFA Rules not processed for user `f69cee9054584c978d8cce8baf004bc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:47.053533 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-6bd58b59-e242-4948-95af-390e98967fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:47.123954 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1310/2633] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:47.128961 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3678ca7-084f-414d-ab5b-c9a07d89f02a admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:47.194572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3678ca7-084f-414d-ab5b-c9a07d89f02a admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:47.199623 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9affd8-652c-4083-90b0-7d66cc79189c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:47.200231 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9affd8-652c-4083-90b0-7d66cc79189c admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:47.200508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9affd8-652c-4083-90b0-7d66cc79189c admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:47.200775 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9affd8-652c-4083-90b0-7d66cc79189c admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:47.201453 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9affd8-652c-4083-90b0-7d66cc79189c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:47.203375 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9affd8-652c-4083-90b0-7d66cc79189c admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:47.233550 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1324/2634] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:19:47 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:19:47.238548 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd58b59-e242-4948-95af-390e98967fa2 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:47.251361 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd58b59-e242-4948-95af-390e98967fa2 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:47.317760 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583b334-16fe-457d-a222-c0b8320afa2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:47.318786 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583b334-16fe-457d-a222-c0b8320afa2a admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:47.319252 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583b334-16fe-457d-a222-c0b8320afa2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:47.319724 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583b334-16fe-457d-a222-c0b8320afa2a admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:47.320893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7583b334-16fe-457d-a222-c0b8320afa2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:47.324880 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7583b334-16fe-457d-a222-c0b8320afa2a admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:47.349091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1311/2635] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:19:47 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:19:47.353708 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9affd8-652c-4083-90b0-7d66cc79189c admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:47.365041 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9affd8-652c-4083-90b0-7d66cc79189c admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:47.370463 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa5fde-d540-4598-a4ff-6d9693491bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:47.371089 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa5fde-d540-4598-a4ff-6d9693491bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:47.371366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa5fde-d540-4598-a4ff-6d9693491bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:47.371636 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa5fde-d540-4598-a4ff-6d9693491bc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:47.372299 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa5fde-d540-4598-a4ff-6d9693491bc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:47.373974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa5fde-d540-4598-a4ff-6d9693491bc4 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:47.375547 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:47.387725 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1325/2636] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:19:47.393005 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583b334-16fe-457d-a222-c0b8320afa2a admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:47.405011 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583b334-16fe-457d-a222-c0b8320afa2a admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:47.411163 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5974dd5-6684-4717-906f-5e627125f708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:47.411875 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5974dd5-6684-4717-906f-5e627125f708 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:47.412316 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5974dd5-6684-4717-906f-5e627125f708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:47.412751 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5974dd5-6684-4717-906f-5e627125f708 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:47.413593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5974dd5-6684-4717-906f-5e627125f708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:47.415767 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5974dd5-6684-4717-906f-5e627125f708 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:47.427612 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1312/2637] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:19:47.431984 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa5fde-d540-4598-a4ff-6d9693491bc4 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:47.443497 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa5fde-d540-4598-a4ff-6d9693491bc4 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:47.449908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8ebb0-7ae1-4394-85f6-0fff08f1422f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:47.450511 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8ebb0-7ae1-4394-85f6-0fff08f1422f admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:47.450784 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8ebb0-7ae1-4394-85f6-0fff08f1422f admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:47.451051 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8ebb0-7ae1-4394-85f6-0fff08f1422f admin admin] PATH_INFO: `/v3/projects/1f8b86795faa42829d4347a6d29ef4f2/users/954fb3aada174a6897701739d1f99a14/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:47.467933 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8ebb0-7ae1-4394-85f6-0fff08f1422f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f8b86795faa42829d4347a6d29ef4f2, user_id=954fb3aada174a6897701739d1f99a14, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:47.470672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8ebb0-7ae1-4394-85f6-0fff08f1422f admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:47.490210 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1326/2638] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:19:47 2022] PUT /identity/v3/projects/1f8b86795faa42829d4347a6d29ef4f2/users/954fb3aada174a6897701739d1f99a14/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:47.504019 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c511e6f-92c2-45b0-9a7c-2d9b1c830d9a None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:47.504332 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c511e6f-92c2-45b0-9a7c-2d9b1c830d9a None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:47.504656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c511e6f-92c2-45b0-9a7c-2d9b1c830d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:47.536734 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-6c511e6f-92c2-45b0-9a7c-2d9b1c830d9a None None] MFA Rules not processed for user `954fb3aada174a6897701739d1f99a14`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:47.537994 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-6c511e6f-92c2-45b0-9a7c-2d9b1c830d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:47.568232 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:47.609826 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1313/2639] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:47.615171 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8ebb0-7ae1-4394-85f6-0fff08f1422f admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:47.681088 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8ebb0-7ae1-4394-85f6-0fff08f1422f admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:47.686422 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fee4d-235f-4816-9751-a74767bd2f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:47.687047 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fee4d-235f-4816-9751-a74767bd2f69 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:47.687314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fee4d-235f-4816-9751-a74767bd2f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:47.687582 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fee4d-235f-4816-9751-a74767bd2f69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:47.688579 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25fee4d-235f-4816-9751-a74767bd2f69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:47.690543 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25fee4d-235f-4816-9751-a74767bd2f69 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:47.723711 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1327/2640] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:19:47 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:19:47.730423 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c511e6f-92c2-45b0-9a7c-2d9b1c830d9a None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:47.742216 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c511e6f-92c2-45b0-9a7c-2d9b1c830d9a None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:47.805631 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0099e05-492c-4752-8511-5fe51aece4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:47.806267 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0099e05-492c-4752-8511-5fe51aece4ad admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:47.806539 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0099e05-492c-4752-8511-5fe51aece4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:47.806809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0099e05-492c-4752-8511-5fe51aece4ad admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:47.807441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0099e05-492c-4752-8511-5fe51aece4ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:47.809553 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0099e05-492c-4752-8511-5fe51aece4ad admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:47.837915 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1314/2641] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:19:47 2022] POST /identity/v3/users => generated 298 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:19:47.841977 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fee4d-235f-4816-9751-a74767bd2f69 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:47.852853 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fee4d-235f-4816-9751-a74767bd2f69 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:47.857723 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1285ad-0de0-474b-8388-e2d0b8ade256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:47.858343 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1285ad-0de0-474b-8388-e2d0b8ade256 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:47.858620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1285ad-0de0-474b-8388-e2d0b8ade256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:47.858889 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1285ad-0de0-474b-8388-e2d0b8ade256 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:47.859572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1285ad-0de0-474b-8388-e2d0b8ade256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:47.861214 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1285ad-0de0-474b-8388-e2d0b8ade256 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:47.872633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1328/2642] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:19:47.877078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0099e05-492c-4752-8511-5fe51aece4ad admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:47.887820 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0099e05-492c-4752-8511-5fe51aece4ad admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:47.892739 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2748c3e-1f78-4772-b963-5df84c486054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:47.893444 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2748c3e-1f78-4772-b963-5df84c486054 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:47.893717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2748c3e-1f78-4772-b963-5df84c486054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:47.893983 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2748c3e-1f78-4772-b963-5df84c486054 admin admin] PATH_INFO: `/v3/projects/ec5ee6f54c9e419ab2faa33351c339a0/users/9ec5d30b9da24356b58cbaa3dc11edce/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:47.914108 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2748c3e-1f78-4772-b963-5df84c486054 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec5ee6f54c9e419ab2faa33351c339a0, user_id=9ec5d30b9da24356b58cbaa3dc11edce, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:47.916791 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2748c3e-1f78-4772-b963-5df84c486054 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:47.931359 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1315/2643] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:19:47 2022] PUT /identity/v3/projects/ec5ee6f54c9e419ab2faa33351c339a0/users/9ec5d30b9da24356b58cbaa3dc11edce/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:47.936113 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1285ad-0de0-474b-8388-e2d0b8ade256 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:47.949770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1285ad-0de0-474b-8388-e2d0b8ade256 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:48.016912 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ed9b1-5b81-425a-82ac-9684d37d247f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:48.017591 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ed9b1-5b81-425a-82ac-9684d37d247f admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:48.017872 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ed9b1-5b81-425a-82ac-9684d37d247f admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:48.018134 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ed9b1-5b81-425a-82ac-9684d37d247f admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:48.018775 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ed9b1-5b81-425a-82ac-9684d37d247f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:48.020434 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ed9b1-5b81-425a-82ac-9684d37d247f admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:48.031732 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1329/2644] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:19:48.037990 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2748c3e-1f78-4772-b963-5df84c486054 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:48.049410 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2748c3e-1f78-4772-b963-5df84c486054 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:48.054436 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7baa29a-9beb-4cd8-b9e7-4693010f0eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:48.055163 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7baa29a-9beb-4cd8-b9e7-4693010f0eac admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:48.055485 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7baa29a-9beb-4cd8-b9e7-4693010f0eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:48.055780 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7baa29a-9beb-4cd8-b9e7-4693010f0eac admin admin] PATH_INFO: `/v3/projects/ec5ee6f54c9e419ab2faa33351c339a0/users/9ec5d30b9da24356b58cbaa3dc11edce/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:48.058011 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7baa29a-9beb-4cd8-b9e7-4693010f0eac admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec5ee6f54c9e419ab2faa33351c339a0, user_id=9ec5d30b9da24356b58cbaa3dc11edce, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:48.060732 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7baa29a-9beb-4cd8-b9e7-4693010f0eac admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:48.074541 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1316/2645] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:19:48 2022] PUT /identity/v3/projects/ec5ee6f54c9e419ab2faa33351c339a0/users/9ec5d30b9da24356b58cbaa3dc11edce/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:48.086960 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d59187-b300-4521-9c6e-fa2ef4b73ead None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:48.087301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d59187-b300-4521-9c6e-fa2ef4b73ead None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:48.087650 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d59187-b300-4521-9c6e-fa2ef4b73ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:48.121929 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-49d59187-b300-4521-9c6e-fa2ef4b73ead None None] MFA Rules not processed for user `9ec5d30b9da24356b58cbaa3dc11edce`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:48.123176 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-49d59187-b300-4521-9c6e-fa2ef4b73ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:48.164232 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:48.210319 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1330/2646] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:48 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:19:48.222904 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7baa29a-9beb-4cd8-b9e7-4693010f0eac admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:48.224998 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-a7baa29a-9beb-4cd8-b9e7-4693010f0eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:48.292816 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7baa29a-9beb-4cd8-b9e7-4693010f0eac admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:48.296870 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4531c1-352b-4d4e-843c-2d1a9efbf422 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:48.297528 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4531c1-352b-4d4e-843c-2d1a9efbf422 service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:48.297799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4531c1-352b-4d4e-843c-2d1a9efbf422 service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:48.298069 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4531c1-352b-4d4e-843c-2d1a9efbf422 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:48.337383 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:48.363875 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4531c1-352b-4d4e-843c-2d1a9efbf422 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:48.365362 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4531c1-352b-4d4e-843c-2d1a9efbf422 service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:48.375344 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1317/2647] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:19:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:19:49.981839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d59187-b300-4521-9c6e-fa2ef4b73ead None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:49.984048 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-49d59187-b300-4521-9c6e-fa2ef4b73ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:50.004025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d59187-b300-4521-9c6e-fa2ef4b73ead None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:50.010016 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67018d13-afbc-440e-a6cd-26b290724040 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:50.012714 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67018d13-afbc-440e-a6cd-26b290724040 service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:50.013250 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67018d13-afbc-440e-a6cd-26b290724040 service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:50.013734 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67018d13-afbc-440e-a6cd-26b290724040 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:50.092101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67018d13-afbc-440e-a6cd-26b290724040 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:50.093596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67018d13-afbc-440e-a6cd-26b290724040 service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:50.104560 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1331/2648] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:19:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:19:50.773150 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4531c1-352b-4d4e-843c-2d1a9efbf422 service cinder] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:50.787571 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4531c1-352b-4d4e-843c-2d1a9efbf422 service cinder] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:50.795022 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29835bf0-f08b-48f2-8c2e-1ba934d9df0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:50.796045 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29835bf0-f08b-48f2-8c2e-1ba934d9df0d service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:50.796521 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29835bf0-f08b-48f2-8c2e-1ba934d9df0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:50.796998 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29835bf0-f08b-48f2-8c2e-1ba934d9df0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:50.808825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29835bf0-f08b-48f2-8c2e-1ba934d9df0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:50.810334 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29835bf0-f08b-48f2-8c2e-1ba934d9df0d service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:50.821698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1318/2649] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:19:50 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:19:54.838428 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67018d13-afbc-440e-a6cd-26b290724040 service cinder] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:54.928929 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67018d13-afbc-440e-a6cd-26b290724040 service cinder] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:54.934249 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debb66e-6e7a-4a82-88f8-40e3deefa3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:54.935429 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debb66e-6e7a-4a82-88f8-40e3deefa3d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:54.935808 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debb66e-6e7a-4a82-88f8-40e3deefa3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:54.936164 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debb66e-6e7a-4a82-88f8-40e3deefa3d5 admin admin] PATH_INFO: `/v3/users/f69cee9054584c978d8cce8baf004bc3` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:54.937705 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debb66e-6e7a-4a82-88f8-40e3deefa3d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f69cee9054584c978d8cce8baf004bc3)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:54.940746 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debb66e-6e7a-4a82-88f8-40e3deefa3d5 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:55.000451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-3debb66e-6e7a-4a82-88f8-40e3deefa3d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f69cee9054584c978d8cce8baf004bc3'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:55.012752 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-3debb66e-6e7a-4a82-88f8-40e3deefa3d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f69cee9054584c978d8cce8baf004bc3'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:55.028077 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:55.032163 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-3debb66e-6e7a-4a82-88f8-40e3deefa3d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f69cee9054584c978d8cce8baf004bc3'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:55.034045 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-3debb66e-6e7a-4a82-88f8-40e3deefa3d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f69cee9054584c978d8cce8baf004bc3'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:55.045623 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-3debb66e-6e7a-4a82-88f8-40e3deefa3d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f69cee9054584c978d8cce8baf004bc3'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:55.055966 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1332/2650] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:19:54 2022] DELETE /identity/v3/users/f69cee9054584c978d8cce8baf004bc3 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:55.062308 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b19a3-d470-4e0c-82b0-52de8b0659fb None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:55.063261 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b19a3-d470-4e0c-82b0-52de8b0659fb None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:55.063261 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b19a3-d470-4e0c-82b0-52de8b0659fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:55.086367 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-3a7b19a3-d470-4e0c-82b0-52de8b0659fb None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:55.087814 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-3a7b19a3-d470-4e0c-82b0-52de8b0659fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:55.160362 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1319/2651] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:55.171649 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debb66e-6e7a-4a82-88f8-40e3deefa3d5 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:55.173322 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-3debb66e-6e7a-4a82-88f8-40e3deefa3d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:55.248375 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debb66e-6e7a-4a82-88f8-40e3deefa3d5 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:55.252626 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc81b5a-ef29-4683-ac2b-152737ba4ef8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:55.253300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc81b5a-ef29-4683-ac2b-152737ba4ef8 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:55.253583 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc81b5a-ef29-4683-ac2b-152737ba4ef8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:55.253850 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc81b5a-ef29-4683-ac2b-152737ba4ef8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:55.265441 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc81b5a-ef29-4683-ac2b-152737ba4ef8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:55.266823 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc81b5a-ef29-4683-ac2b-152737ba4ef8 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:55.278286 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1333/2652] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:19:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:19:55.643106 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b19a3-d470-4e0c-82b0-52de8b0659fb None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:55.644757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-3a7b19a3-d470-4e0c-82b0-52de8b0659fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:55.660038 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b19a3-d470-4e0c-82b0-52de8b0659fb None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:55.666208 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b397c9-434a-4f5a-bcf5-befecf40610f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:55.667184 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b397c9-434a-4f5a-bcf5-befecf40610f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:55.667648 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b397c9-434a-4f5a-bcf5-befecf40610f admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:55.667992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b397c9-434a-4f5a-bcf5-befecf40610f admin admin] PATH_INFO: `/v3/projects/763fd6e7f5604e229987f55abf4130a0` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:55.669498 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b397c9-434a-4f5a-bcf5-befecf40610f admin admin] RBAC: Authorizing `identity:delete_project(project_id=763fd6e7f5604e229987f55abf4130a0)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:55.672754 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b397c9-434a-4f5a-bcf5-befecf40610f admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:55.739055 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-82b397c9-434a-4f5a-bcf5-befecf40610f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '763fd6e7f5604e229987f55abf4130a0', 'request_id': 'req-82b397c9-434a-4f5a-bcf5-befecf40610f'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:55.747816 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-82b397c9-434a-4f5a-bcf5-befecf40610f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '763fd6e7f5604e229987f55abf4130a0', 'request_id': 'req-82b397c9-434a-4f5a-bcf5-befecf40610f'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:55.754452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-82b397c9-434a-4f5a-bcf5-befecf40610f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '763fd6e7f5604e229987f55abf4130a0', 'request_id': 'req-82b397c9-434a-4f5a-bcf5-befecf40610f'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:55.767034 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-82b397c9-434a-4f5a-bcf5-befecf40610f admin admin] The token cache is being invalidate because project 763fd6e7f5604e229987f55abf4130a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:19:55.767332 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-82b397c9-434a-4f5a-bcf5-befecf40610f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:55.768121 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-82b397c9-434a-4f5a-bcf5-befecf40610f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:55.769884 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1320/2653] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:19:55 2022] DELETE /identity/v3/projects/763fd6e7f5604e229987f55abf4130a0 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:55.774596 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc81b5a-ef29-4683-ac2b-152737ba4ef8 service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:55.787177 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc81b5a-ef29-4683-ac2b-152737ba4ef8 service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:55.875879 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55f4e2-855a-4b3f-b9c0-e84aecab1c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:55.876618 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55f4e2-855a-4b3f-b9c0-e84aecab1c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:55.876763 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55f4e2-855a-4b3f-b9c0-e84aecab1c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:55.876976 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55f4e2-855a-4b3f-b9c0-e84aecab1c30 admin admin] PATH_INFO: `/v3/users/954fb3aada174a6897701739d1f99a14` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:55.878371 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55f4e2-855a-4b3f-b9c0-e84aecab1c30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=954fb3aada174a6897701739d1f99a14)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:55.880347 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55f4e2-855a-4b3f-b9c0-e84aecab1c30 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:55.939611 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2f55f4e2-855a-4b3f-b9c0-e84aecab1c30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '954fb3aada174a6897701739d1f99a14'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:55.949321 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2f55f4e2-855a-4b3f-b9c0-e84aecab1c30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '954fb3aada174a6897701739d1f99a14'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:55.964786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2f55f4e2-855a-4b3f-b9c0-e84aecab1c30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '954fb3aada174a6897701739d1f99a14'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:55.965685 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2f55f4e2-855a-4b3f-b9c0-e84aecab1c30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '954fb3aada174a6897701739d1f99a14'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:55.975084 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-2f55f4e2-855a-4b3f-b9c0-e84aecab1c30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '954fb3aada174a6897701739d1f99a14'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:55.982260 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1334/2654] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:19:55 2022] DELETE /identity/v3/users/954fb3aada174a6897701739d1f99a14 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:56.303358 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b397c9-434a-4f5a-bcf5-befecf40610f admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:56.305828 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-82b397c9-434a-4f5a-bcf5-befecf40610f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:56.334512 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b397c9-434a-4f5a-bcf5-befecf40610f admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:56.385528 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:56.402521 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e438dd-b14d-4802-9ef2-bdccfdd3c1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:56.403076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e438dd-b14d-4802-9ef2-bdccfdd3c1d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:56.403352 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e438dd-b14d-4802-9ef2-bdccfdd3c1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:56.403620 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e438dd-b14d-4802-9ef2-bdccfdd3c1d5 admin admin] PATH_INFO: `/v3/projects/1f8b86795faa42829d4347a6d29ef4f2` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:56.404880 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e438dd-b14d-4802-9ef2-bdccfdd3c1d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f8b86795faa42829d4347a6d29ef4f2)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:56.406899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e438dd-b14d-4802-9ef2-bdccfdd3c1d5 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:56.471542 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-18e438dd-b14d-4802-9ef2-bdccfdd3c1d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f8b86795faa42829d4347a6d29ef4f2', 'request_id': 'req-18e438dd-b14d-4802-9ef2-bdccfdd3c1d5'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:56.481686 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-18e438dd-b14d-4802-9ef2-bdccfdd3c1d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f8b86795faa42829d4347a6d29ef4f2', 'request_id': 'req-18e438dd-b14d-4802-9ef2-bdccfdd3c1d5'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:56.489659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-18e438dd-b14d-4802-9ef2-bdccfdd3c1d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f8b86795faa42829d4347a6d29ef4f2', 'request_id': 'req-18e438dd-b14d-4802-9ef2-bdccfdd3c1d5'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:56.502053 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-18e438dd-b14d-4802-9ef2-bdccfdd3c1d5 admin admin] The token cache is being invalidate because project 1f8b86795faa42829d4347a6d29ef4f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:19:56.502347 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-18e438dd-b14d-4802-9ef2-bdccfdd3c1d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:56.502999 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-18e438dd-b14d-4802-9ef2-bdccfdd3c1d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:56.504549 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1321/2655] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:19:56 2022] DELETE /identity/v3/projects/1f8b86795faa42829d4347a6d29ef4f2 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:56.509233 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55f4e2-855a-4b3f-b9c0-e84aecab1c30 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:56.511013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-2f55f4e2-855a-4b3f-b9c0-e84aecab1c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:56.525050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55f4e2-855a-4b3f-b9c0-e84aecab1c30 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:56.529492 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:56.594441 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f699ce-027e-4006-9e9d-fe2c4965eaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:56.595101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f699ce-027e-4006-9e9d-fe2c4965eaa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:56.595378 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f699ce-027e-4006-9e9d-fe2c4965eaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:56.595638 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f699ce-027e-4006-9e9d-fe2c4965eaa1 admin admin] PATH_INFO: `/v3/users/9ec5d30b9da24356b58cbaa3dc11edce` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:56.596856 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f699ce-027e-4006-9e9d-fe2c4965eaa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ec5d30b9da24356b58cbaa3dc11edce)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:56.598855 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f699ce-027e-4006-9e9d-fe2c4965eaa1 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:56.657374 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-86f699ce-027e-4006-9e9d-fe2c4965eaa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ec5d30b9da24356b58cbaa3dc11edce'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:56.666998 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-86f699ce-027e-4006-9e9d-fe2c4965eaa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ec5d30b9da24356b58cbaa3dc11edce'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:56.682897 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-86f699ce-027e-4006-9e9d-fe2c4965eaa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ec5d30b9da24356b58cbaa3dc11edce'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:56.683773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-86f699ce-027e-4006-9e9d-fe2c4965eaa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ec5d30b9da24356b58cbaa3dc11edce'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:56.693430 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-86f699ce-027e-4006-9e9d-fe2c4965eaa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ec5d30b9da24356b58cbaa3dc11edce'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:56.700860 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1335/2656] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:19:56 2022] DELETE /identity/v3/users/9ec5d30b9da24356b58cbaa3dc11edce => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:57.141474 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e438dd-b14d-4802-9ef2-bdccfdd3c1d5 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:57.143204 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-18e438dd-b14d-4802-9ef2-bdccfdd3c1d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:57.157888 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e438dd-b14d-4802-9ef2-bdccfdd3c1d5 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:57.228919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3f38b-b709-4aff-b5cb-39afc18ec10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:57.229573 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3f38b-b709-4aff-b5cb-39afc18ec10c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:57.229871 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3f38b-b709-4aff-b5cb-39afc18ec10c admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:57.230139 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3f38b-b709-4aff-b5cb-39afc18ec10c admin admin] PATH_INFO: `/v3/projects/ec5ee6f54c9e419ab2faa33351c339a0` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:57.231396 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a3f38b-b709-4aff-b5cb-39afc18ec10c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec5ee6f54c9e419ab2faa33351c339a0)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:57.233393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a3f38b-b709-4aff-b5cb-39afc18ec10c admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:57.294465 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-18a3f38b-b709-4aff-b5cb-39afc18ec10c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec5ee6f54c9e419ab2faa33351c339a0', 'request_id': 'req-18a3f38b-b709-4aff-b5cb-39afc18ec10c'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:57.303781 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-18a3f38b-b709-4aff-b5cb-39afc18ec10c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec5ee6f54c9e419ab2faa33351c339a0', 'request_id': 'req-18a3f38b-b709-4aff-b5cb-39afc18ec10c'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:57.311221 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-18a3f38b-b709-4aff-b5cb-39afc18ec10c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec5ee6f54c9e419ab2faa33351c339a0', 'request_id': 'req-18a3f38b-b709-4aff-b5cb-39afc18ec10c'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:57.326170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-18a3f38b-b709-4aff-b5cb-39afc18ec10c admin admin] The token cache is being invalidate because project ec5ee6f54c9e419ab2faa33351c339a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:19:57.326535 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-18a3f38b-b709-4aff-b5cb-39afc18ec10c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:57.327465 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-18a3f38b-b709-4aff-b5cb-39afc18ec10c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:19:57.329219 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1322/2657] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:19:57 2022] DELETE /identity/v3/projects/ec5ee6f54c9e419ab2faa33351c339a0 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:57.334877 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c3b2a-62b9-4aaf-89f0-4f2150089322 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:57.335512 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c3b2a-62b9-4aaf-89f0-4f2150089322 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:57.335512 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c3b2a-62b9-4aaf-89f0-4f2150089322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:57.358401 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-105c3b2a-62b9-4aaf-89f0-4f2150089322 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:57.359739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-105c3b2a-62b9-4aaf-89f0-4f2150089322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:57.431112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1336/2658] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:57.439617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976e368-ec59-416f-9fd2-b4e36dad5540 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:57.439885 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976e368-ec59-416f-9fd2-b4e36dad5540 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:57.440146 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976e368-ec59-416f-9fd2-b4e36dad5540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:57.456227 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:57.463826 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-a976e368-ec59-416f-9fd2-b4e36dad5540 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:57.466125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-a976e368-ec59-416f-9fd2-b4e36dad5540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:57.475601 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1323/2659] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:57.480711 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c3b2a-62b9-4aaf-89f0-4f2150089322 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:57.491541 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c3b2a-62b9-4aaf-89f0-4f2150089322 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:57.497049 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9b487-68aa-4ccf-b5a4-2e10f13ec8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:57.497723 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9b487-68aa-4ccf-b5a4-2e10f13ec8b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:57.498003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9b487-68aa-4ccf-b5a4-2e10f13ec8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:57.498271 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9b487-68aa-4ccf-b5a4-2e10f13ec8b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:57.498946 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e9b487-68aa-4ccf-b5a4-2e10f13ec8b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:19:57.499248 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e9b487-68aa-4ccf-b5a4-2e10f13ec8b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:57.500901 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e9b487-68aa-4ccf-b5a4-2e10f13ec8b6 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:57.516620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1337/2660] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:19:57.521567 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a976e368-ec59-416f-9fd2-b4e36dad5540 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:57.531860 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a976e368-ec59-416f-9fd2-b4e36dad5540 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:57.537634 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858f379-cf4e-4441-a1f6-b0ea543375f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:57.538380 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858f379-cf4e-4441-a1f6-b0ea543375f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:57.538695 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858f379-cf4e-4441-a1f6-b0ea543375f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:57.539057 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858f379-cf4e-4441-a1f6-b0ea543375f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:57.540061 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858f379-cf4e-4441-a1f6-b0ea543375f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:57.542210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858f379-cf4e-4441-a1f6-b0ea543375f3 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:57.575526 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1324/2661] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:19:57 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:19:57.580521 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9b487-68aa-4ccf-b5a4-2e10f13ec8b6 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:57.592488 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9b487-68aa-4ccf-b5a4-2e10f13ec8b6 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:57.603723 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:19:57.673542 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93c9f6-35d5-49e7-aae1-d207c8db0144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:57.674292 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93c9f6-35d5-49e7-aae1-d207c8db0144 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:57.674611 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93c9f6-35d5-49e7-aae1-d207c8db0144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:57.674955 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93c9f6-35d5-49e7-aae1-d207c8db0144 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:57.675662 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93c9f6-35d5-49e7-aae1-d207c8db0144 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:57.677878 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93c9f6-35d5-49e7-aae1-d207c8db0144 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:57.703221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1338/2662] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:19:57 2022] POST /identity/v3/users => generated 287 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:19:57.707857 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858f379-cf4e-4441-a1f6-b0ea543375f3 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:57.719353 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858f379-cf4e-4441-a1f6-b0ea543375f3 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:57.725197 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452bea7-cc62-47b7-9f9f-00520584bb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:57.725957 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452bea7-cc62-47b7-9f9f-00520584bb2e admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:57.726271 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452bea7-cc62-47b7-9f9f-00520584bb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:57.726582 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452bea7-cc62-47b7-9f9f-00520584bb2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:57.727285 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f452bea7-cc62-47b7-9f9f-00520584bb2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:57.729028 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f452bea7-cc62-47b7-9f9f-00520584bb2e admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:57.741618 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1325/2663] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:19:57.746472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93c9f6-35d5-49e7-aae1-d207c8db0144 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:57.757184 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93c9f6-35d5-49e7-aae1-d207c8db0144 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:57.762482 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96af060-326c-48c8-a6e3-59a2763f091e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:57.763240 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96af060-326c-48c8-a6e3-59a2763f091e admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:57.763601 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96af060-326c-48c8-a6e3-59a2763f091e admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:57.763914 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96af060-326c-48c8-a6e3-59a2763f091e admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:57.764626 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96af060-326c-48c8-a6e3-59a2763f091e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:57.766554 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96af060-326c-48c8-a6e3-59a2763f091e admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:57.778809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1339/2664] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:19:57.783468 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452bea7-cc62-47b7-9f9f-00520584bb2e admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:57.794365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452bea7-cc62-47b7-9f9f-00520584bb2e admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:57.799263 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e521e9-5ec9-402c-9a42-8444f692f3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:57.799949 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e521e9-5ec9-402c-9a42-8444f692f3df admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:57.800215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e521e9-5ec9-402c-9a42-8444f692f3df admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:57.800478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e521e9-5ec9-402c-9a42-8444f692f3df admin admin] PATH_INFO: `/v3/projects/8b5fece5b881463091697f7dbe3988f4/users/b2b5486855554f388e6ee753e748161b/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:57.818859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e521e9-5ec9-402c-9a42-8444f692f3df admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b5fece5b881463091697f7dbe3988f4, user_id=b2b5486855554f388e6ee753e748161b, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:57.821600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e521e9-5ec9-402c-9a42-8444f692f3df admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:57.839087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1326/2665] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:19:57 2022] PUT /identity/v3/projects/8b5fece5b881463091697f7dbe3988f4/users/b2b5486855554f388e6ee753e748161b/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:19:57.844948 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07665159-84a7-4450-8441-85768ab65610 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:57.845264 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07665159-84a7-4450-8441-85768ab65610 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:57.845535 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07665159-84a7-4450-8441-85768ab65610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:57.866627 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-07665159-84a7-4450-8441-85768ab65610 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:57.867880 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-07665159-84a7-4450-8441-85768ab65610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:57.943489 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1340/2666] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:57.956215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e521e9-5ec9-402c-9a42-8444f692f3df admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:57.958828 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f3e521e9-5ec9-402c-9a42-8444f692f3df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:58.051540 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e521e9-5ec9-402c-9a42-8444f692f3df admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:58.056326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdae57c4-282d-4c16-869d-8985192ee916 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:58.057023 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae57c4-282d-4c16-869d-8985192ee916 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:58.057455 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae57c4-282d-4c16-869d-8985192ee916 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:58.057832 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae57c4-282d-4c16-869d-8985192ee916 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:58.069866 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdae57c4-282d-4c16-869d-8985192ee916 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:58.071499 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdae57c4-282d-4c16-869d-8985192ee916 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:58.083786 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1327/2667] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:19:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:19:58.849284 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8740f11-b13c-4b5a-bf9a-e9288269d05c None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:58.849881 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8740f11-b13c-4b5a-bf9a-e9288269d05c None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:58.850212 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8740f11-b13c-4b5a-bf9a-e9288269d05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:58.895322 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-e8740f11-b13c-4b5a-bf9a-e9288269d05c None None] MFA Rules not processed for user `b2b5486855554f388e6ee753e748161b`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:19:58.897923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-e8740f11-b13c-4b5a-bf9a-e9288269d05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:59.030148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1341/2668] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:19:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:19:59.041987 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdae57c4-282d-4c16-869d-8985192ee916 service neutron] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:19:59.044209 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-cdae57c4-282d-4c16-869d-8985192ee916 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:19:59.124688 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdae57c4-282d-4c16-869d-8985192ee916 service neutron] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:19:59.128551 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628a8a1-cf13-485a-8bc5-5418d5881579 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:19:59.129215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628a8a1-cf13-485a-8bc5-5418d5881579 service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:19:59.129496 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628a8a1-cf13-485a-8bc5-5418d5881579 service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:19:59.129774 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628a8a1-cf13-485a-8bc5-5418d5881579 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:19:59.139852 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628a8a1-cf13-485a-8bc5-5418d5881579 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:19:59.141367 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628a8a1-cf13-485a-8bc5-5418d5881579 service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:19:59.152106 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1328/2669] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:19:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:20:00.636143 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e766ef-4834-4054-9da8-ec9f4e3a1327 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:20:00.636769 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e766ef-4834-4054-9da8-ec9f4e3a1327 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:20:00.636769 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e766ef-4834-4054-9da8-ec9f4e3a1327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:20:00.658380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-d1e766ef-4834-4054-9da8-ec9f4e3a1327 None None] MFA Rules not processed for user `b2b5486855554f388e6ee753e748161b`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:20:00.659697 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-d1e766ef-4834-4054-9da8-ec9f4e3a1327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:20:00.668398 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1342/2670] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:20:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:20:01.001235 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628a8a1-cf13-485a-8bc5-5418d5881579 service cinder] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:20:01.015426 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628a8a1-cf13-485a-8bc5-5418d5881579 service cinder] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:20:01.019832 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29967d6f-f899-42c7-84a0-7c439a355e8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:20:01.020569 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29967d6f-f899-42c7-84a0-7c439a355e8f service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:20:01.020850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29967d6f-f899-42c7-84a0-7c439a355e8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:20:01.021114 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29967d6f-f899-42c7-84a0-7c439a355e8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:20:01.023378 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-29967d6f-f899-42c7-84a0-7c439a355e8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:20:01.035193 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29967d6f-f899-42c7-84a0-7c439a355e8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:20:01.036700 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29967d6f-f899-42c7-84a0-7c439a355e8f service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:20:01.046737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1329/2671] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:20:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:20:10.652195 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=90,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:20:11.041967 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=82,keystone:INSERT=2 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:20:15.286055 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673f36b-8cde-4240-b2e3-43b747f69834 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:20:15.288039 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673f36b-8cde-4240-b2e3-43b747f69834 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:20:15.288313 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673f36b-8cde-4240-b2e3-43b747f69834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:20:15.307892 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-2673f36b-8cde-4240-b2e3-43b747f69834 None None] MFA Rules not processed for user `b2b5486855554f388e6ee753e748161b`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:20:15.309096 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-2673f36b-8cde-4240-b2e3-43b747f69834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:20:15.316585 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1343/2672] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:20:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:20:25.302064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:20:36.678694 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0260dde-228c-42d6-9e00-c637a92b067a None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:20:36.679255 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0260dde-228c-42d6-9e00-c637a92b067a None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:20:36.679255 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0260dde-228c-42d6-9e00-c637a92b067a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:20:36.730093 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-c0260dde-228c-42d6-9e00-c637a92b067a None None] MFA Rules not processed for user `3fada78ceaeb4f3bad77c516f6dfcfd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:20:36.731581 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c0260dde-228c-42d6-9e00-c637a92b067a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:20:36.833970 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1330/2673] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 15:20:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:20:36.892036 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673f36b-8cde-4240-b2e3-43b747f69834 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:20:36.893606 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-2673f36b-8cde-4240-b2e3-43b747f69834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:20:37.004205 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673f36b-8cde-4240-b2e3-43b747f69834 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:20:37.010127 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9899b5-82aa-4217-be72-0eb3d2c4d423 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05aa5e40ee4542719657bc03b2015e72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:20:37.010766 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9899b5-82aa-4217-be72-0eb3d2c4d423 service swift] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:20:37.011037 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9899b5-82aa-4217-be72-0eb3d2c4d423 service swift] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:20:37.011299 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9899b5-82aa-4217-be72-0eb3d2c4d423 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:20:37.020683 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9899b5-82aa-4217-be72-0eb3d2c4d423 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:20:37.022299 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9899b5-82aa-4217-be72-0eb3d2c4d423 service swift] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:20:37.030703 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1344/2674] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 15:20:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 15:20:46.831117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=45 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:20:47.027084 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=47 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:20:52.643411 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70538974-c5d0-4419-9efa-fea6bfb2c609 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:20:52.644022 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70538974-c5d0-4419-9efa-fea6bfb2c609 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:20:52.644022 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70538974-c5d0-4419-9efa-fea6bfb2c609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:20:52.665797 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-70538974-c5d0-4419-9efa-fea6bfb2c609 None None] MFA Rules not processed for user `3fada78ceaeb4f3bad77c516f6dfcfd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:20:52.666980 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-70538974-c5d0-4419-9efa-fea6bfb2c609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:20:52.675316 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1331/2675] 10.222.0.44 () {58 vars in 967 bytes} [Sun Sep 11 15:20:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:20:52.689595 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9899b5-82aa-4217-be72-0eb3d2c4d423 service swift] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:20:52.703875 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9899b5-82aa-4217-be72-0eb3d2c4d423 service swift] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:20:52.709692 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d341c-6e57-4b9d-9d40-c6c719d56a91 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05aa5e40ee4542719657bc03b2015e72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:20:52.710288 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d341c-6e57-4b9d-9d40-c6c719d56a91 service swift] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:20:52.710564 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d341c-6e57-4b9d-9d40-c6c719d56a91 service swift] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:20:52.710829 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d341c-6e57-4b9d-9d40-c6c719d56a91 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:20:52.719355 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d341c-6e57-4b9d-9d40-c6c719d56a91 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:20:52.720924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d341c-6e57-4b9d-9d40-c6c719d56a91 service swift] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:20:52.728982 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1345/2676] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 15:20:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 15:20:54.654309 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70538974-c5d0-4419-9efa-fea6bfb2c609 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:20:54.655866 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-70538974-c5d0-4419-9efa-fea6bfb2c609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:20:54.723241 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d341c-6e57-4b9d-9d40-c6c719d56a91 service swift] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:20:54.757380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d341c-6e57-4b9d-9d40-c6c719d56a91 service swift] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:20:54.757644 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70538974-c5d0-4419-9efa-fea6bfb2c609 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:20:54.762984 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01843ba1-4017-4f9d-8097-326050315c54 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead06362f8a2444bbfe1405cd602f1c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:20:54.763604 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01843ba1-4017-4f9d-8097-326050315c54 service placement] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:20:54.763904 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01843ba1-4017-4f9d-8097-326050315c54 service placement] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:20:54.764037 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64daa3d1-7526-4d30-a9d7-178989084455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:20:54.764161 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01843ba1-4017-4f9d-8097-326050315c54 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:20:54.764769 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64daa3d1-7526-4d30-a9d7-178989084455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:20:54.765157 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64daa3d1-7526-4d30-a9d7-178989084455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:20:54.765435 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64daa3d1-7526-4d30-a9d7-178989084455 admin admin] PATH_INFO: `/v3/users/b2b5486855554f388e6ee753e748161b` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:20:54.766050 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-01843ba1-4017-4f9d-8097-326050315c54 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:20:54.766571 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64daa3d1-7526-4d30-a9d7-178989084455 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2b5486855554f388e6ee753e748161b)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:20:54.768486 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64daa3d1-7526-4d30-a9d7-178989084455 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:20:54.778553 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01843ba1-4017-4f9d-8097-326050315c54 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:20:54.780250 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01843ba1-4017-4f9d-8097-326050315c54 service placement] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:20:54.790962 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1332/2677] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:20:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:20:54.830671 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-64daa3d1-7526-4d30-a9d7-178989084455 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2b5486855554f388e6ee753e748161b'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:20:54.841159 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-64daa3d1-7526-4d30-a9d7-178989084455 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2b5486855554f388e6ee753e748161b'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:20:54.857078 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-64daa3d1-7526-4d30-a9d7-178989084455 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2b5486855554f388e6ee753e748161b'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:20:54.858111 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-64daa3d1-7526-4d30-a9d7-178989084455 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2b5486855554f388e6ee753e748161b'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:20:54.866943 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-64daa3d1-7526-4d30-a9d7-178989084455 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2b5486855554f388e6ee753e748161b'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:20:54.875146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1346/2678] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:20:54 2022] DELETE /identity/v3/users/b2b5486855554f388e6ee753e748161b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:20:55.067863 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01843ba1-4017-4f9d-8097-326050315c54 service placement] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:20:55.082104 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01843ba1-4017-4f9d-8097-326050315c54 service placement] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:20:55.155942 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c490c798-b3d2-4f96-98ba-7260cff870db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:20:55.156597 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490c798-b3d2-4f96-98ba-7260cff870db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:20:55.156903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490c798-b3d2-4f96-98ba-7260cff870db admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:20:55.157356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490c798-b3d2-4f96-98ba-7260cff870db admin admin] PATH_INFO: `/v3/projects/8b5fece5b881463091697f7dbe3988f4` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:20:55.158631 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c490c798-b3d2-4f96-98ba-7260cff870db admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b5fece5b881463091697f7dbe3988f4)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:20:55.160677 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c490c798-b3d2-4f96-98ba-7260cff870db admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:20:55.218527 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c490c798-b3d2-4f96-98ba-7260cff870db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b5fece5b881463091697f7dbe3988f4', 'request_id': 'req-c490c798-b3d2-4f96-98ba-7260cff870db'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:20:55.227799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c490c798-b3d2-4f96-98ba-7260cff870db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b5fece5b881463091697f7dbe3988f4', 'request_id': 'req-c490c798-b3d2-4f96-98ba-7260cff870db'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:20:55.233896 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c490c798-b3d2-4f96-98ba-7260cff870db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b5fece5b881463091697f7dbe3988f4', 'request_id': 'req-c490c798-b3d2-4f96-98ba-7260cff870db'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:20:55.234904 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:20:55.261978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c490c798-b3d2-4f96-98ba-7260cff870db admin admin] The token cache is being invalidate because project 8b5fece5b881463091697f7dbe3988f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:20:55.262233 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c490c798-b3d2-4f96-98ba-7260cff870db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:20:55.262881 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-c490c798-b3d2-4f96-98ba-7260cff870db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:20:55.264437 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1333/2679] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:20:55 2022] DELETE /identity/v3/projects/8b5fece5b881463091697f7dbe3988f4 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:20:55.269758 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235f18a-bb87-4ccb-a480-8333b5ccd8b5 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:20:55.270044 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235f18a-bb87-4ccb-a480-8333b5ccd8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:20:55.270314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235f18a-bb87-4ccb-a480-8333b5ccd8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:20:55.333752 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-d235f18a-bb87-4ccb-a480-8333b5ccd8b5 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:20:55.334970 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-d235f18a-bb87-4ccb-a480-8333b5ccd8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:20:55.402868 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1347/2680] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:20:55.510604 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ee731-8b97-4f13-906e-20ccb4304dfb None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:20:55.510885 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ee731-8b97-4f13-906e-20ccb4304dfb None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:20:55.511178 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ee731-8b97-4f13-906e-20ccb4304dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:20:55.533197 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-596ee731-8b97-4f13-906e-20ccb4304dfb None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:20:55.534570 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-596ee731-8b97-4f13-906e-20ccb4304dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:20:55.543492 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1334/2681] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:20:55.548198 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235f18a-bb87-4ccb-a480-8333b5ccd8b5 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:20:55.558353 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235f18a-bb87-4ccb-a480-8333b5ccd8b5 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:20:55.564142 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cabbf2e-6485-407d-9581-b91ef1fbfaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:20:55.564775 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabbf2e-6485-407d-9581-b91ef1fbfaaa admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:20:55.565049 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabbf2e-6485-407d-9581-b91ef1fbfaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:20:55.565358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cabbf2e-6485-407d-9581-b91ef1fbfaaa admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:20:55.566018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cabbf2e-6485-407d-9581-b91ef1fbfaaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:20:55.566318 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cabbf2e-6485-407d-9581-b91ef1fbfaaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:20:55.567929 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cabbf2e-6485-407d-9581-b91ef1fbfaaa admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:20:55.578013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:20:55.582573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1348/2682] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:20:55.587206 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ee731-8b97-4f13-906e-20ccb4304dfb None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:20:55.596944 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ee731-8b97-4f13-906e-20ccb4304dfb None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:20:55.601677 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf3239-3b42-469c-8997-fee69fb22753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:20:55.602310 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf3239-3b42-469c-8997-fee69fb22753 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:20:55.602582 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf3239-3b42-469c-8997-fee69fb22753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:20:55.602845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf3239-3b42-469c-8997-fee69fb22753 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:20:55.603478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebf3239-3b42-469c-8997-fee69fb22753 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:20:55.605519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebf3239-3b42-469c-8997-fee69fb22753 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:20:55.639882 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1335/2683] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:20:55 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:20:55.644662 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cabbf2e-6485-407d-9581-b91ef1fbfaaa admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:20:55.658564 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cabbf2e-6485-407d-9581-b91ef1fbfaaa admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:20:55.725271 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e2bee-bcf3-4da2-be0e-ad0c378c0c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:20:55.726101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e2bee-bcf3-4da2-be0e-ad0c378c0c42 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:20:55.726211 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e2bee-bcf3-4da2-be0e-ad0c378c0c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:20:55.726473 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e2bee-bcf3-4da2-be0e-ad0c378c0c42 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:20:55.727119 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e2bee-bcf3-4da2-be0e-ad0c378c0c42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:20:55.729040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e2bee-bcf3-4da2-be0e-ad0c378c0c42 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:20:55.753001 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1349/2684] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:20:55 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:20:55.757126 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf3239-3b42-469c-8997-fee69fb22753 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:20:55.769004 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf3239-3b42-469c-8997-fee69fb22753 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:20:55.774927 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68efed9-d1b0-4cc2-b342-65332af9bd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:20:55.775581 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68efed9-d1b0-4cc2-b342-65332af9bd34 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:20:55.775850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68efed9-d1b0-4cc2-b342-65332af9bd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:20:55.776112 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68efed9-d1b0-4cc2-b342-65332af9bd34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:20:55.776758 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68efed9-d1b0-4cc2-b342-65332af9bd34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:20:55.778841 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68efed9-d1b0-4cc2-b342-65332af9bd34 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:20:55.790107 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1336/2685] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:20:55.794396 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e2bee-bcf3-4da2-be0e-ad0c378c0c42 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:20:55.805249 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e2bee-bcf3-4da2-be0e-ad0c378c0c42 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:20:55.810728 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933264c-9dbc-42b3-9bf5-16b1ef9e6b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:20:55.811787 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933264c-9dbc-42b3-9bf5-16b1ef9e6b01 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:20:55.812096 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933264c-9dbc-42b3-9bf5-16b1ef9e6b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:20:55.812406 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933264c-9dbc-42b3-9bf5-16b1ef9e6b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:20:55.813161 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933264c-9dbc-42b3-9bf5-16b1ef9e6b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:20:55.815058 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933264c-9dbc-42b3-9bf5-16b1ef9e6b01 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:20:55.828462 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1350/2686] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:20:55.833022 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68efed9-d1b0-4cc2-b342-65332af9bd34 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:20:55.844633 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68efed9-d1b0-4cc2-b342-65332af9bd34 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:20:55.851333 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781a71d-969d-4f50-a4f6-fda9d9be8be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:20:55.852054 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781a71d-969d-4f50-a4f6-fda9d9be8be2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:20:55.852518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781a71d-969d-4f50-a4f6-fda9d9be8be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:20:55.852793 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781a71d-969d-4f50-a4f6-fda9d9be8be2 admin admin] PATH_INFO: `/v3/projects/ae1e34e58001410699b1cb3fd88057e5/users/faf283b28e864d15b316edcf281e96f9/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:20:55.870856 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b781a71d-969d-4f50-a4f6-fda9d9be8be2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae1e34e58001410699b1cb3fd88057e5, user_id=faf283b28e864d15b316edcf281e96f9, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:20:55.873775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b781a71d-969d-4f50-a4f6-fda9d9be8be2 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:20:55.891064 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1337/2687] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:20:55 2022] PUT /identity/v3/projects/ae1e34e58001410699b1cb3fd88057e5/users/faf283b28e864d15b316edcf281e96f9/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:20:55.901985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c0d97-c5eb-460a-ada5-0c79ae407c46 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:20:55.902397 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c0d97-c5eb-460a-ada5-0c79ae407c46 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:20:55.902760 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c0d97-c5eb-460a-ada5-0c79ae407c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:20:55.934936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-132c0d97-c5eb-460a-ada5-0c79ae407c46 None None] MFA Rules not processed for user `faf283b28e864d15b316edcf281e96f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:20:55.936528 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-132c0d97-c5eb-460a-ada5-0c79ae407c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:20:56.007279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1351/2688] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:20:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:20:56.018788 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781a71d-969d-4f50-a4f6-fda9d9be8be2 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:20:56.020718 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b781a71d-969d-4f50-a4f6-fda9d9be8be2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:20:56.110100 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781a71d-969d-4f50-a4f6-fda9d9be8be2 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:20:56.114870 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebc548-f947-441d-a757-d7c3d49360a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:20:56.115707 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebc548-f947-441d-a757-d7c3d49360a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:20:56.116095 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebc548-f947-441d-a757-d7c3d49360a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:20:56.116420 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebc548-f947-441d-a757-d7c3d49360a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:20:56.128078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ebc548-f947-441d-a757-d7c3d49360a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:20:56.129824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ebc548-f947-441d-a757-d7c3d49360a5 service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:20:56.140759 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1338/2689] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:20:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:20:57.621248 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3b246-69e2-4b26-b6f0-5055d71aff84 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:20:57.622148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3b246-69e2-4b26-b6f0-5055d71aff84 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:20:57.622317 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3b246-69e2-4b26-b6f0-5055d71aff84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:20:57.643656 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-0bd3b246-69e2-4b26-b6f0-5055d71aff84 None None] MFA Rules not processed for user `faf283b28e864d15b316edcf281e96f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:20:57.644943 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-0bd3b246-69e2-4b26-b6f0-5055d71aff84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:20:57.652488 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1352/2690] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:20:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:20:59.532187 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebc548-f947-441d-a757-d7c3d49360a5 service cinder] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:20:59.533680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b1ebc548-f947-441d-a757-d7c3d49360a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:20:59.632616 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:20:59.638312 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebc548-f947-441d-a757-d7c3d49360a5 service cinder] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:20:59.647001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43ee45-2642-45ab-8895-bb689ead3488 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead06362f8a2444bbfe1405cd602f1c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:20:59.647623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43ee45-2642-45ab-8895-bb689ead3488 service placement] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:20:59.647947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43ee45-2642-45ab-8895-bb689ead3488 service placement] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:20:59.648441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43ee45-2642-45ab-8895-bb689ead3488 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:20:59.650099 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-4a43ee45-2642-45ab-8895-bb689ead3488 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:20:59.668476 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43ee45-2642-45ab-8895-bb689ead3488 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:20:59.671505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43ee45-2642-45ab-8895-bb689ead3488 service placement] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:20:59.685963 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1339/2691] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:20:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:21:04.821895 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe36619-c027-479f-806b-79c98601622c None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:04.822835 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe36619-c027-479f-806b-79c98601622c None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:04.822835 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe36619-c027-479f-806b-79c98601622c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:04.846346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-bfe36619-c027-479f-806b-79c98601622c None None] MFA Rules not processed for user `948640729c5c4975b3b050129eb63eea`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:21:04.847624 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-bfe36619-c027-479f-806b-79c98601622c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:21:04.922325 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1353/2692] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:21:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:21:09.679737 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=4 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:21:14.914463 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=94,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:21:16.052849 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c0f7a-e21a-4883-991f-274cde2a0ac2 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:16.053494 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c0f7a-e21a-4883-991f-274cde2a0ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:16.053494 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c0f7a-e21a-4883-991f-274cde2a0ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:16.080986 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-c93c0f7a-e21a-4883-991f-274cde2a0ac2 None None] MFA Rules not processed for user `3fada78ceaeb4f3bad77c516f6dfcfd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:21:16.083798 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c93c0f7a-e21a-4883-991f-274cde2a0ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:21:16.156621 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1340/2693] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 15:21:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:21:16.176834 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe36619-c027-479f-806b-79c98601622c None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:21:16.177637 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-bfe36619-c027-479f-806b-79c98601622c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:21:16.277637 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe36619-c027-479f-806b-79c98601622c None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:21:16.283019 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9aa85b-125d-4be5-a82b-e4448860fa00 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05aa5e40ee4542719657bc03b2015e72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:21:16.283654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9aa85b-125d-4be5-a82b-e4448860fa00 service swift] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:16.283930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9aa85b-125d-4be5-a82b-e4448860fa00 service swift] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:16.284674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9aa85b-125d-4be5-a82b-e4448860fa00 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:16.296063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9aa85b-125d-4be5-a82b-e4448860fa00 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:21:16.297734 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9aa85b-125d-4be5-a82b-e4448860fa00 service swift] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:21:16.313338 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1354/2694] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 15:21:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 15:21:16.898998 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30098724-2539-41f2-af39-8e9ce9564b86 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:16.899328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30098724-2539-41f2-af39-8e9ce9564b86 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:16.899599 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30098724-2539-41f2-af39-8e9ce9564b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:16.920768 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-30098724-2539-41f2-af39-8e9ce9564b86 None None] MFA Rules not processed for user `3fada78ceaeb4f3bad77c516f6dfcfd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:21:16.922054 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-30098724-2539-41f2-af39-8e9ce9564b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:21:16.929946 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1341/2695] 10.222.0.44 () {58 vars in 967 bytes} [Sun Sep 11 15:21:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:21:16.945504 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9aa85b-125d-4be5-a82b-e4448860fa00 service swift] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:21:16.956846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9aa85b-125d-4be5-a82b-e4448860fa00 service swift] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:21:16.963028 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78c7d4-e364-45a1-a272-d9c31ff68b8c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05aa5e40ee4542719657bc03b2015e72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:21:16.963625 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78c7d4-e364-45a1-a272-d9c31ff68b8c service swift] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:16.963948 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78c7d4-e364-45a1-a272-d9c31ff68b8c service swift] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:16.964211 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78c7d4-e364-45a1-a272-d9c31ff68b8c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:16.974198 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78c7d4-e364-45a1-a272-d9c31ff68b8c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:21:16.975773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78c7d4-e364-45a1-a272-d9c31ff68b8c service swift] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:21:16.984686 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1355/2696] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 15:21:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 15:21:17.238048 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c13cf6-5656-483b-aa01-ac32a95bb912 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:17.238762 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c13cf6-5656-483b-aa01-ac32a95bb912 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:17.238762 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c13cf6-5656-483b-aa01-ac32a95bb912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:17.265724 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-70c13cf6-5656-483b-aa01-ac32a95bb912 None None] MFA Rules not processed for user `3fada78ceaeb4f3bad77c516f6dfcfd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:21:17.267170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-70c13cf6-5656-483b-aa01-ac32a95bb912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:21:17.283954 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1342/2697] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 15:21:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:21:17.300312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78c7d4-e364-45a1-a272-d9c31ff68b8c service swift] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:21:17.310307 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78c7d4-e364-45a1-a272-d9c31ff68b8c service swift] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:21:17.316140 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7c449-dc60-4e0f-b401-980534b32d41 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05aa5e40ee4542719657bc03b2015e72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:21:17.316746 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7c449-dc60-4e0f-b401-980534b32d41 service swift] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:17.317022 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7c449-dc60-4e0f-b401-980534b32d41 service swift] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:17.317322 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7c449-dc60-4e0f-b401-980534b32d41 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:17.326484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f7c449-dc60-4e0f-b401-980534b32d41 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:21:17.328123 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f7c449-dc60-4e0f-b401-980534b32d41 service swift] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:21:17.336546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1356/2698] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 15:21:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 15:21:17.706738 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b569e-977b-468a-8796-b7e763dd1da0 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:17.707025 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b569e-977b-468a-8796-b7e763dd1da0 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:17.707300 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b569e-977b-468a-8796-b7e763dd1da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:17.729891 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-c01b569e-977b-468a-8796-b7e763dd1da0 None None] MFA Rules not processed for user `3fada78ceaeb4f3bad77c516f6dfcfd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:21:17.731222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c01b569e-977b-468a-8796-b7e763dd1da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:21:17.740137 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1343/2699] 10.222.0.44 () {58 vars in 967 bytes} [Sun Sep 11 15:21:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:21:17.753434 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7c449-dc60-4e0f-b401-980534b32d41 service swift] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:21:17.764179 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7c449-dc60-4e0f-b401-980534b32d41 service swift] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:21:17.769439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25656b-3b6a-4bb7-b1ef-4109cb78bd88 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05aa5e40ee4542719657bc03b2015e72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:21:17.770048 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25656b-3b6a-4bb7-b1ef-4109cb78bd88 service swift] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:17.770328 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25656b-3b6a-4bb7-b1ef-4109cb78bd88 service swift] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:17.770602 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25656b-3b6a-4bb7-b1ef-4109cb78bd88 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:17.782560 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25656b-3b6a-4bb7-b1ef-4109cb78bd88 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:21:17.784442 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25656b-3b6a-4bb7-b1ef-4109cb78bd88 service swift] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:21:17.797514 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1357/2700] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 15:21:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 15:21:20.022367 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b569e-977b-468a-8796-b7e763dd1da0 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:21:20.092428 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b569e-977b-468a-8796-b7e763dd1da0 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:21:20.097636 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b5e8b6-5630-4688-89e5-6c343f3482d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:21:20.098491 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5e8b6-5630-4688-89e5-6c343f3482d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:20.098768 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5e8b6-5630-4688-89e5-6c343f3482d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:20.099035 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5e8b6-5630-4688-89e5-6c343f3482d1 admin admin] PATH_INFO: `/v3/users/faf283b28e864d15b316edcf281e96f9` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:20.100455 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b5e8b6-5630-4688-89e5-6c343f3482d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=faf283b28e864d15b316edcf281e96f9)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:21:20.103022 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b5e8b6-5630-4688-89e5-6c343f3482d1 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:21:20.155411 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-80b5e8b6-5630-4688-89e5-6c343f3482d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faf283b28e864d15b316edcf281e96f9'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:21:20.170688 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-80b5e8b6-5630-4688-89e5-6c343f3482d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faf283b28e864d15b316edcf281e96f9'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:21:20.185379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-80b5e8b6-5630-4688-89e5-6c343f3482d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faf283b28e864d15b316edcf281e96f9'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:21:20.186633 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-80b5e8b6-5630-4688-89e5-6c343f3482d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faf283b28e864d15b316edcf281e96f9'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:21:20.194384 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-80b5e8b6-5630-4688-89e5-6c343f3482d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faf283b28e864d15b316edcf281e96f9'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:21:20.201272 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1344/2701] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:21:20 2022] DELETE /identity/v3/users/faf283b28e864d15b316edcf281e96f9 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:21:20.205822 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671c71e-572d-42e6-a72a-6d4c83c6f61d None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:20.206326 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671c71e-572d-42e6-a72a-6d4c83c6f61d None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:20.206447 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671c71e-572d-42e6-a72a-6d4c83c6f61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:20.225674 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-5671c71e-572d-42e6-a72a-6d4c83c6f61d None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:21:20.226914 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-5671c71e-572d-42e6-a72a-6d4c83c6f61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:21:20.294617 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1358/2702] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:21:20.304541 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b5e8b6-5630-4688-89e5-6c343f3482d1 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:21:20.306012 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-80b5e8b6-5630-4688-89e5-6c343f3482d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:21:20.312349 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:21:20.393419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b5e8b6-5630-4688-89e5-6c343f3482d1 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:21:20.397193 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b943008-6de1-4a88-9944-0a0a8af51bb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:21:20.397979 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b943008-6de1-4a88-9944-0a0a8af51bb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:20.398281 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b943008-6de1-4a88-9944-0a0a8af51bb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:20.398389 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b943008-6de1-4a88-9944-0a0a8af51bb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:20.410201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b943008-6de1-4a88-9944-0a0a8af51bb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:21:20.411674 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b943008-6de1-4a88-9944-0a0a8af51bb1 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:21:20.422752 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1345/2703] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:21:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:21:20.608895 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671c71e-572d-42e6-a72a-6d4c83c6f61d None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:21:20.610445 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-5671c71e-572d-42e6-a72a-6d4c83c6f61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:21:20.628893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671c71e-572d-42e6-a72a-6d4c83c6f61d None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:21:20.634811 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb0870-582c-4d72-a9b1-4193270c7c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:21:20.635604 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb0870-582c-4d72-a9b1-4193270c7c96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:20.636121 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb0870-582c-4d72-a9b1-4193270c7c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:20.636403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb0870-582c-4d72-a9b1-4193270c7c96 admin admin] PATH_INFO: `/v3/projects/ae1e34e58001410699b1cb3fd88057e5` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:20.638067 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb0870-582c-4d72-a9b1-4193270c7c96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae1e34e58001410699b1cb3fd88057e5)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:21:20.640444 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb0870-582c-4d72-a9b1-4193270c7c96 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:21:20.688912 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:21:20.692304 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-1dbb0870-582c-4d72-a9b1-4193270c7c96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae1e34e58001410699b1cb3fd88057e5', 'request_id': 'req-1dbb0870-582c-4d72-a9b1-4193270c7c96'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:21:20.699229 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-1dbb0870-582c-4d72-a9b1-4193270c7c96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae1e34e58001410699b1cb3fd88057e5', 'request_id': 'req-1dbb0870-582c-4d72-a9b1-4193270c7c96'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:21:20.706028 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-1dbb0870-582c-4d72-a9b1-4193270c7c96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae1e34e58001410699b1cb3fd88057e5', 'request_id': 'req-1dbb0870-582c-4d72-a9b1-4193270c7c96'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:21:20.719875 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-1dbb0870-582c-4d72-a9b1-4193270c7c96 admin admin] The token cache is being invalidate because project ae1e34e58001410699b1cb3fd88057e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:21:20.720164 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-1dbb0870-582c-4d72-a9b1-4193270c7c96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:21:20.720800 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-1dbb0870-582c-4d72-a9b1-4193270c7c96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:21:20.722323 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1359/2704] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:21:20 2022] DELETE /identity/v3/projects/ae1e34e58001410699b1cb3fd88057e5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:21:20.727381 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc15eb-0ad5-425f-b68e-ef3ccb635514 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:20.727653 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc15eb-0ad5-425f-b68e-ef3ccb635514 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:20.727916 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc15eb-0ad5-425f-b68e-ef3ccb635514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:20.747830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-eadc15eb-0ad5-425f-b68e-ef3ccb635514 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:21:20.749025 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-eadc15eb-0ad5-425f-b68e-ef3ccb635514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:21:20.813740 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1346/2705] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:21:20.821899 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb6f2e-0d73-4e6e-970d-f534adf409d0 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:20.822188 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb6f2e-0d73-4e6e-970d-f534adf409d0 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:20.822503 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb6f2e-0d73-4e6e-970d-f534adf409d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:20.846898 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-cceb6f2e-0d73-4e6e-970d-f534adf409d0 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:21:20.848093 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-cceb6f2e-0d73-4e6e-970d-f534adf409d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:21:20.856683 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1360/2706] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:21:20.861004 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadc15eb-0ad5-425f-b68e-ef3ccb635514 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:21:20.871457 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadc15eb-0ad5-425f-b68e-ef3ccb635514 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:21:20.877537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24cdb7d-583b-4115-a2c9-1b1fa424b782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:21:20.878137 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cdb7d-583b-4115-a2c9-1b1fa424b782 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:20.878429 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cdb7d-583b-4115-a2c9-1b1fa424b782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:20.878685 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cdb7d-583b-4115-a2c9-1b1fa424b782 admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:20.879324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24cdb7d-583b-4115-a2c9-1b1fa424b782 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:21:20.879633 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24cdb7d-583b-4115-a2c9-1b1fa424b782 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:21:20.881249 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24cdb7d-583b-4115-a2c9-1b1fa424b782 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:21:20.893512 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1347/2707] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:21:20.900002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb6f2e-0d73-4e6e-970d-f534adf409d0 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:21:20.909730 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb6f2e-0d73-4e6e-970d-f534adf409d0 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:21:20.914602 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd50046-775c-49ed-ad05-3f1a0749cd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:21:20.915195 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd50046-775c-49ed-ad05-3f1a0749cd6a admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:20.915465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd50046-775c-49ed-ad05-3f1a0749cd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:20.915729 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd50046-775c-49ed-ad05-3f1a0749cd6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:20.916344 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd50046-775c-49ed-ad05-3f1a0749cd6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:21:20.918346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd50046-775c-49ed-ad05-3f1a0749cd6a admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:21:20.948696 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1361/2708] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:21:20 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:21:20.953045 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24cdb7d-583b-4115-a2c9-1b1fa424b782 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:21:20.964061 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24cdb7d-583b-4115-a2c9-1b1fa424b782 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:21:21.043318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2099b-f69b-4038-8c2c-6b017dd9c049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:21:21.044017 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2099b-f69b-4038-8c2c-6b017dd9c049 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:21.044179 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2099b-f69b-4038-8c2c-6b017dd9c049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:21.044473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2099b-f69b-4038-8c2c-6b017dd9c049 admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:21.045100 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2099b-f69b-4038-8c2c-6b017dd9c049 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:21:21.047063 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2099b-f69b-4038-8c2c-6b017dd9c049 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:21:21.071947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1348/2709] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:21:20 2022] POST /identity/v3/users => generated 286 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:21:21.076313 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd50046-775c-49ed-ad05-3f1a0749cd6a admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:21:21.086904 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd50046-775c-49ed-ad05-3f1a0749cd6a admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:21:21.091874 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e7d5e-7c88-4573-98e3-8c002423ddf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:21:21.092492 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e7d5e-7c88-4573-98e3-8c002423ddf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:21.092770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e7d5e-7c88-4573-98e3-8c002423ddf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:21.093042 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e7d5e-7c88-4573-98e3-8c002423ddf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:21.093733 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e7d5e-7c88-4573-98e3-8c002423ddf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:21:21.095330 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e7d5e-7c88-4573-98e3-8c002423ddf1 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:21:21.106170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1362/2710] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:21:21.110365 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2099b-f69b-4038-8c2c-6b017dd9c049 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:21:21.117149 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:21:21.121376 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2099b-f69b-4038-8c2c-6b017dd9c049 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:21:21.127232 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc1183-011e-489e-9302-cb5e16f20ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:21:21.127845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc1183-011e-489e-9302-cb5e16f20ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:21.128124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc1183-011e-489e-9302-cb5e16f20ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:21.128388 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc1183-011e-489e-9302-cb5e16f20ad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:21.129035 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc1183-011e-489e-9302-cb5e16f20ad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:21:21.130789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc1183-011e-489e-9302-cb5e16f20ad9 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:21:21.143426 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1349/2711] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:21:21.148657 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e7d5e-7c88-4573-98e3-8c002423ddf1 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:21:21.159261 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e7d5e-7c88-4573-98e3-8c002423ddf1 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:21:21.164839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec73484-3c33-4a60-857b-06a33c6e005e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:21:21.165577 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec73484-3c33-4a60-857b-06a33c6e005e admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:21.165836 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec73484-3c33-4a60-857b-06a33c6e005e admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:21.166108 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec73484-3c33-4a60-857b-06a33c6e005e admin admin] PATH_INFO: `/v3/projects/ae3105c3aa57472899713ef172a862e4/users/6e253dad57f44ab2955c353792fd5097/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:21.183485 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec73484-3c33-4a60-857b-06a33c6e005e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae3105c3aa57472899713ef172a862e4, user_id=6e253dad57f44ab2955c353792fd5097, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:21:21.186830 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec73484-3c33-4a60-857b-06a33c6e005e admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:21:21.203299 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1363/2712] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:21:21 2022] PUT /identity/v3/projects/ae3105c3aa57472899713ef172a862e4/users/6e253dad57f44ab2955c353792fd5097/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:21:21.213448 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f22a3-e3f9-4f8e-b2d8-10f056f7befb None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:21.213963 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f22a3-e3f9-4f8e-b2d8-10f056f7befb None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:21.214449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f22a3-e3f9-4f8e-b2d8-10f056f7befb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:21.240650 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-d90f22a3-e3f9-4f8e-b2d8-10f056f7befb None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:21:21.242220 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-d90f22a3-e3f9-4f8e-b2d8-10f056f7befb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:21:21.306519 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1350/2713] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:21:21.316789 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec73484-3c33-4a60-857b-06a33c6e005e admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:21:21.318747 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-cec73484-3c33-4a60-857b-06a33c6e005e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:21:21.513369 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec73484-3c33-4a60-857b-06a33c6e005e admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:21:21.517068 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110aff39-7a4e-438b-80d9-2bb833c0963e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:21:21.517710 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110aff39-7a4e-438b-80d9-2bb833c0963e service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:21.517981 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110aff39-7a4e-438b-80d9-2bb833c0963e service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:21.518242 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110aff39-7a4e-438b-80d9-2bb833c0963e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:21.528381 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110aff39-7a4e-438b-80d9-2bb833c0963e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:21:21.529830 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110aff39-7a4e-438b-80d9-2bb833c0963e service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:21:21.540235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1364/2714] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:21:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:21:25.845296 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b04f74-ac09-459e-bcac-d0a888122e2d None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:25.845851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b04f74-ac09-459e-bcac-d0a888122e2d None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:25.845851 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b04f74-ac09-459e-bcac-d0a888122e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:25.875349 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-b2b04f74-ac09-459e-bcac-d0a888122e2d None None] MFA Rules not processed for user `6e253dad57f44ab2955c353792fd5097`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:21:25.876592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b2b04f74-ac09-459e-bcac-d0a888122e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:21:25.974407 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1351/2715] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:21:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:21:25.980743 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110aff39-7a4e-438b-80d9-2bb833c0963e service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:21:25.997740 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110aff39-7a4e-438b-80d9-2bb833c0963e service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:21:26.008846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee316b7-3030-43e8-bf6b-417588e71614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:21:26.009498 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee316b7-3030-43e8-bf6b-417588e71614 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:26.009788 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee316b7-3030-43e8-bf6b-417588e71614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:26.010044 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee316b7-3030-43e8-bf6b-417588e71614 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:26.010767 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee316b7-3030-43e8-bf6b-417588e71614 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:21:26.012787 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee316b7-3030-43e8-bf6b-417588e71614 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:21:26.056295 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1365/2716] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:21:25 2022] POST /identity/v3/projects => generated 371 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:21:26.061733 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b04f74-ac09-459e-bcac-d0a888122e2d None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:21:26.073399 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b04f74-ac09-459e-bcac-d0a888122e2d None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:21:26.142854 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568369e4-7ddb-4930-bce8-3b326206d68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:21:26.143746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568369e4-7ddb-4930-bce8-3b326206d68c admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:26.144032 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568369e4-7ddb-4930-bce8-3b326206d68c admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:26.144521 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568369e4-7ddb-4930-bce8-3b326206d68c admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:26.145310 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568369e4-7ddb-4930-bce8-3b326206d68c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:21:26.147207 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568369e4-7ddb-4930-bce8-3b326206d68c admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:21:26.182143 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1352/2717] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:21:26 2022] POST /identity/v3/users => generated 292 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:21:26.182386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee316b7-3030-43e8-bf6b-417588e71614 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:21:26.206843 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee316b7-3030-43e8-bf6b-417588e71614 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:21:26.212703 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb38ed5-e411-4e6a-baf6-7e25a48f020d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:21:26.213381 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb38ed5-e411-4e6a-baf6-7e25a48f020d admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:26.213682 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb38ed5-e411-4e6a-baf6-7e25a48f020d admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:26.214162 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb38ed5-e411-4e6a-baf6-7e25a48f020d admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:26.214869 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb38ed5-e411-4e6a-baf6-7e25a48f020d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:21:26.216459 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb38ed5-e411-4e6a-baf6-7e25a48f020d admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:21:26.232703 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1366/2718] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:21:26.237264 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568369e4-7ddb-4930-bce8-3b326206d68c admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:21:26.248747 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568369e4-7ddb-4930-bce8-3b326206d68c admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:21:26.259562 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea088c-94ef-48b1-bc37-6a7fd44bf8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:21:26.260469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea088c-94ef-48b1-bc37-6a7fd44bf8c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:26.260809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea088c-94ef-48b1-bc37-6a7fd44bf8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:26.261163 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea088c-94ef-48b1-bc37-6a7fd44bf8c3 admin admin] PATH_INFO: `/v3/projects/864272140621484fbb0ffa59274d101e/users/cf0ef0a9c79d4b4b83ec78f51e304972/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:26.278534 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:21:26.287414 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea088c-94ef-48b1-bc37-6a7fd44bf8c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=864272140621484fbb0ffa59274d101e, user_id=cf0ef0a9c79d4b4b83ec78f51e304972, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:21:26.292928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea088c-94ef-48b1-bc37-6a7fd44bf8c3 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:21:26.314062 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1353/2719] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:21:26 2022] PUT /identity/v3/projects/864272140621484fbb0ffa59274d101e/users/cf0ef0a9c79d4b4b83ec78f51e304972/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:21:26.319456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb38ed5-e411-4e6a-baf6-7e25a48f020d admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:21:26.330679 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb38ed5-e411-4e6a-baf6-7e25a48f020d admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:21:26.374054 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:21:26.396677 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63173a8c-6ca8-4ee7-a787-26661e9ddddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:21:26.397402 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63173a8c-6ca8-4ee7-a787-26661e9ddddb admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:26.397673 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63173a8c-6ca8-4ee7-a787-26661e9ddddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:26.398066 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63173a8c-6ca8-4ee7-a787-26661e9ddddb admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:26.398742 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63173a8c-6ca8-4ee7-a787-26661e9ddddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:21:26.400364 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63173a8c-6ca8-4ee7-a787-26661e9ddddb admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:21:26.412579 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1367/2720] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:21:26.417361 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea088c-94ef-48b1-bc37-6a7fd44bf8c3 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:21:26.428119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea088c-94ef-48b1-bc37-6a7fd44bf8c3 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:21:26.433245 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ccaedc-8d6a-4a84-a4f7-0eafbced5ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:21:26.433823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ccaedc-8d6a-4a84-a4f7-0eafbced5ec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:26.434094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ccaedc-8d6a-4a84-a4f7-0eafbced5ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:26.434361 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ccaedc-8d6a-4a84-a4f7-0eafbced5ec7 admin admin] PATH_INFO: `/v3/projects/864272140621484fbb0ffa59274d101e/users/cf0ef0a9c79d4b4b83ec78f51e304972/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:26.436749 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ccaedc-8d6a-4a84-a4f7-0eafbced5ec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=864272140621484fbb0ffa59274d101e, user_id=cf0ef0a9c79d4b4b83ec78f51e304972, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:21:26.439441 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ccaedc-8d6a-4a84-a4f7-0eafbced5ec7 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:21:26.453055 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1354/2721] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:21:26 2022] PUT /identity/v3/projects/864272140621484fbb0ffa59274d101e/users/cf0ef0a9c79d4b4b83ec78f51e304972/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:21:31.023613 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6fca5-d853-4501-a3ee-0646000b45ad None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:31.024495 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6fca5-d853-4501-a3ee-0646000b45ad None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:31.024924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6fca5-d853-4501-a3ee-0646000b45ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:31.056138 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-87a6fca5-d853-4501-a3ee-0646000b45ad None None] MFA Rules not processed for user `cf0ef0a9c79d4b4b83ec78f51e304972`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:21:31.057623 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-87a6fca5-d853-4501-a3ee-0646000b45ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:21:31.158704 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1368/2722] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:21:31 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:21:31.169902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ccaedc-8d6a-4a84-a4f7-0eafbced5ec7 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:21:31.172834 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-48ccaedc-8d6a-4a84-a4f7-0eafbced5ec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:21:31.328348 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ccaedc-8d6a-4a84-a4f7-0eafbced5ec7 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:21:31.339748 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029f806-d680-46af-84f4-d4cf05e0b1ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:21:31.340401 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029f806-d680-46af-84f4-d4cf05e0b1ae service nova] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:31.340674 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029f806-d680-46af-84f4-d4cf05e0b1ae service nova] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:31.340941 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029f806-d680-46af-84f4-d4cf05e0b1ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:31.536983 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2029f806-d680-46af-84f4-d4cf05e0b1ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:21:31.538910 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2029f806-d680-46af-84f4-d4cf05e0b1ae service nova] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:21:31.549195 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1355/2723] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:21:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:21:32.649723 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c1337-b09a-4f09-a3bb-2081d0d4cdb3 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:21:32.650282 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c1337-b09a-4f09-a3bb-2081d0d4cdb3 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:21:32.650282 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c1337-b09a-4f09-a3bb-2081d0d4cdb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:21:32.671572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-260c1337-b09a-4f09-a3bb-2081d0d4cdb3 None None] MFA Rules not processed for user `6e253dad57f44ab2955c353792fd5097`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:21:32.673812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-260c1337-b09a-4f09-a3bb-2081d0d4cdb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:21:32.683181 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1369/2724] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:21:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:21:41.544854 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=74,keystone:INSERT=2 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:21:42.665700 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=48 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:22:04.609844 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029f806-d680-46af-84f4-d4cf05e0b1ae service nova] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:22:04.677105 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029f806-d680-46af-84f4-d4cf05e0b1ae service nova] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:22:04.680931 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34542c5b-9802-4bef-b052-be1ac89c90cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:22:04.681581 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34542c5b-9802-4bef-b052-be1ac89c90cd service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:22:04.681849 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34542c5b-9802-4bef-b052-be1ac89c90cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:22:04.682111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34542c5b-9802-4bef-b052-be1ac89c90cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:22:04.694150 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34542c5b-9802-4bef-b052-be1ac89c90cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:22:04.695585 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34542c5b-9802-4bef-b052-be1ac89c90cd service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:22:04.705979 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1356/2725] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:22:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:22:14.701685 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=26 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:22:29.207369 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c1337-b09a-4f09-a3bb-2081d0d4cdb3 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:22:29.209729 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-260c1337-b09a-4f09-a3bb-2081d0d4cdb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:22:29.296474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c1337-b09a-4f09-a3bb-2081d0d4cdb3 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:22:29.301024 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef836470-a7ed-42e8-867d-4e68ea806474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:22:29.301712 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef836470-a7ed-42e8-867d-4e68ea806474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:22:29.301991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef836470-a7ed-42e8-867d-4e68ea806474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:22:29.302266 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef836470-a7ed-42e8-867d-4e68ea806474 admin admin] PATH_INFO: `/v3/users/948640729c5c4975b3b050129eb63eea` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:22:29.303386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef836470-a7ed-42e8-867d-4e68ea806474 admin admin] RBAC: Authorizing `identity:delete_user(user_id=948640729c5c4975b3b050129eb63eea)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:22:29.305324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef836470-a7ed-42e8-867d-4e68ea806474 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:22:29.355120 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-ef836470-a7ed-42e8-867d-4e68ea806474 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '948640729c5c4975b3b050129eb63eea'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:22:29.363931 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-ef836470-a7ed-42e8-867d-4e68ea806474 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '948640729c5c4975b3b050129eb63eea'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:22:29.376333 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-ef836470-a7ed-42e8-867d-4e68ea806474 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '948640729c5c4975b3b050129eb63eea'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:22:29.377018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-ef836470-a7ed-42e8-867d-4e68ea806474 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '948640729c5c4975b3b050129eb63eea'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:22:29.383888 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-ef836470-a7ed-42e8-867d-4e68ea806474 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '948640729c5c4975b3b050129eb63eea'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:22:29.389791 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1370/2726] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:22:29 2022] DELETE /identity/v3/users/948640729c5c4975b3b050129eb63eea => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:22:29.588317 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34542c5b-9802-4bef-b052-be1ac89c90cd service neutron] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:22:29.599398 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34542c5b-9802-4bef-b052-be1ac89c90cd service neutron] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:22:29.655875 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3440f18c-3a7f-4aa8-811e-bfe9f3cd4787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:22:29.656499 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3440f18c-3a7f-4aa8-811e-bfe9f3cd4787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:22:29.656828 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3440f18c-3a7f-4aa8-811e-bfe9f3cd4787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:22:29.657093 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3440f18c-3a7f-4aa8-811e-bfe9f3cd4787 admin admin] PATH_INFO: `/v3/projects/bf1611f722d34ce0b058da277806b326` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:22:29.658220 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3440f18c-3a7f-4aa8-811e-bfe9f3cd4787 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf1611f722d34ce0b058da277806b326)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:22:29.660124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3440f18c-3a7f-4aa8-811e-bfe9f3cd4787 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:22:29.709959 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3440f18c-3a7f-4aa8-811e-bfe9f3cd4787 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf1611f722d34ce0b058da277806b326', 'request_id': 'req-3440f18c-3a7f-4aa8-811e-bfe9f3cd4787'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:22:29.718846 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3440f18c-3a7f-4aa8-811e-bfe9f3cd4787 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf1611f722d34ce0b058da277806b326', 'request_id': 'req-3440f18c-3a7f-4aa8-811e-bfe9f3cd4787'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:22:29.725614 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3440f18c-3a7f-4aa8-811e-bfe9f3cd4787 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf1611f722d34ce0b058da277806b326', 'request_id': 'req-3440f18c-3a7f-4aa8-811e-bfe9f3cd4787'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:22:29.739397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3440f18c-3a7f-4aa8-811e-bfe9f3cd4787 admin admin] The token cache is being invalidate because project bf1611f722d34ce0b058da277806b326 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:22:29.739688 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3440f18c-3a7f-4aa8-811e-bfe9f3cd4787 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:22:29.740514 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3440f18c-3a7f-4aa8-811e-bfe9f3cd4787 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:22:29.742731 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1357/2727] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:22:29 2022] DELETE /identity/v3/projects/bf1611f722d34ce0b058da277806b326 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:22:29.748057 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7aece-0251-4645-afff-e4e091b7eea8 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:22:29.748335 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7aece-0251-4645-afff-e4e091b7eea8 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:22:29.748606 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7aece-0251-4645-afff-e4e091b7eea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:22:29.769518 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-67c7aece-0251-4645-afff-e4e091b7eea8 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:22:29.770758 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-67c7aece-0251-4645-afff-e4e091b7eea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:22:29.845395 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1371/2728] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:22:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:22:29.853579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f10e9-f584-4d07-8ba9-595b24a85037 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:22:29.853854 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f10e9-f584-4d07-8ba9-595b24a85037 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:22:29.854119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f10e9-f584-4d07-8ba9-595b24a85037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:22:29.875083 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-603f10e9-f584-4d07-8ba9-595b24a85037 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:22:29.876236 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-603f10e9-f584-4d07-8ba9-595b24a85037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:22:29.888090 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1358/2729] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:22:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:22:29.893470 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7aece-0251-4645-afff-e4e091b7eea8 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:22:29.903222 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7aece-0251-4645-afff-e4e091b7eea8 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:22:29.908025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d51fec-ee0f-4359-ab19-7334f48fec8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:22:29.908620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d51fec-ee0f-4359-ab19-7334f48fec8f admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:22:29.908894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d51fec-ee0f-4359-ab19-7334f48fec8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:22:29.909167 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d51fec-ee0f-4359-ab19-7334f48fec8f admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:22:29.909824 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d51fec-ee0f-4359-ab19-7334f48fec8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:22:29.910121 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d51fec-ee0f-4359-ab19-7334f48fec8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:22:29.911711 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d51fec-ee0f-4359-ab19-7334f48fec8f admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:22:29.923269 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1372/2730] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:22:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:22:29.928351 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603f10e9-f584-4d07-8ba9-595b24a85037 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:22:29.943844 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603f10e9-f584-4d07-8ba9-595b24a85037 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:22:29.951202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf417b9-9e1e-435a-9bdb-3be3f629617d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:22:29.952063 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf417b9-9e1e-435a-9bdb-3be3f629617d admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:22:29.952402 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf417b9-9e1e-435a-9bdb-3be3f629617d admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:22:29.952790 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf417b9-9e1e-435a-9bdb-3be3f629617d admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:22:29.953859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf417b9-9e1e-435a-9bdb-3be3f629617d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:22:29.955785 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf417b9-9e1e-435a-9bdb-3be3f629617d admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:22:29.996268 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1359/2731] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:22:29 2022] POST /identity/v3/projects => generated 371 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:22:30.002003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d51fec-ee0f-4359-ab19-7334f48fec8f admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:22:30.012453 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d51fec-ee0f-4359-ab19-7334f48fec8f admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:22:30.016165 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:22:30.073648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0365847-189e-4432-bc1a-31105723f8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:22:30.074244 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0365847-189e-4432-bc1a-31105723f8e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:22:30.074531 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0365847-189e-4432-bc1a-31105723f8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:22:30.074799 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0365847-189e-4432-bc1a-31105723f8e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:22:30.075419 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0365847-189e-4432-bc1a-31105723f8e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:22:30.077326 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0365847-189e-4432-bc1a-31105723f8e3 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:22:30.101448 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1373/2732] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:22:30 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:22:30.105885 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf417b9-9e1e-435a-9bdb-3be3f629617d admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:22:30.116971 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf417b9-9e1e-435a-9bdb-3be3f629617d admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:22:30.122414 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd7b97-75b9-4236-b824-7e2cfe22aa04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:22:30.123044 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd7b97-75b9-4236-b824-7e2cfe22aa04 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:22:30.123318 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd7b97-75b9-4236-b824-7e2cfe22aa04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:22:30.123584 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd7b97-75b9-4236-b824-7e2cfe22aa04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:22:30.124879 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd7b97-75b9-4236-b824-7e2cfe22aa04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:22:30.126581 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd7b97-75b9-4236-b824-7e2cfe22aa04 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:22:30.137604 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1360/2733] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:22:30.143111 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0365847-189e-4432-bc1a-31105723f8e3 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:22:30.153586 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0365847-189e-4432-bc1a-31105723f8e3 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:22:30.158403 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf65048-9bab-4a9e-b765-a674d2ef6866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:22:30.159010 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf65048-9bab-4a9e-b765-a674d2ef6866 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:22:30.159283 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf65048-9bab-4a9e-b765-a674d2ef6866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:22:30.159546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf65048-9bab-4a9e-b765-a674d2ef6866 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:22:30.160172 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf65048-9bab-4a9e-b765-a674d2ef6866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:22:30.161787 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf65048-9bab-4a9e-b765-a674d2ef6866 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:22:30.172655 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1374/2734] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:22:30.177053 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd7b97-75b9-4236-b824-7e2cfe22aa04 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:22:30.189435 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd7b97-75b9-4236-b824-7e2cfe22aa04 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:22:30.194643 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732042c-e4fe-413a-89c5-28694d6b0d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:22:30.195331 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732042c-e4fe-413a-89c5-28694d6b0d96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:22:30.195607 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732042c-e4fe-413a-89c5-28694d6b0d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:22:30.195876 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732042c-e4fe-413a-89c5-28694d6b0d96 admin admin] PATH_INFO: `/v3/projects/277522ac9b0e4dac8c312060a69ec295/users/644f9e8b80a54fc0a970e3077b9b8155/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:22:30.212177 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9732042c-e4fe-413a-89c5-28694d6b0d96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=277522ac9b0e4dac8c312060a69ec295, user_id=644f9e8b80a54fc0a970e3077b9b8155, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:22:30.214909 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9732042c-e4fe-413a-89c5-28694d6b0d96 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:22:30.230201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1361/2735] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:22:30 2022] PUT /identity/v3/projects/277522ac9b0e4dac8c312060a69ec295/users/644f9e8b80a54fc0a970e3077b9b8155/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:22:30.240880 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d164093-560f-4df4-8048-43ce7e114adc None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:22:30.241504 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d164093-560f-4df4-8048-43ce7e114adc None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:22:30.241504 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d164093-560f-4df4-8048-43ce7e114adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:22:30.272520 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-0d164093-560f-4df4-8048-43ce7e114adc None None] MFA Rules not processed for user `644f9e8b80a54fc0a970e3077b9b8155`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:22:30.273788 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-0d164093-560f-4df4-8048-43ce7e114adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:22:30.346227 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1375/2736] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:22:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:22:30.351261 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732042c-e4fe-413a-89c5-28694d6b0d96 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:22:30.373055 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:22:30.426436 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732042c-e4fe-413a-89c5-28694d6b0d96 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:22:30.431625 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce0fa43-bf6b-4cf0-9c9b-0b57feff9ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:22:30.432229 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0fa43-bf6b-4cf0-9c9b-0b57feff9ef4 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:22:30.432503 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0fa43-bf6b-4cf0-9c9b-0b57feff9ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:22:30.432819 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0fa43-bf6b-4cf0-9c9b-0b57feff9ef4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:22:30.433470 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce0fa43-bf6b-4cf0-9c9b-0b57feff9ef4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:22:30.435859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce0fa43-bf6b-4cf0-9c9b-0b57feff9ef4 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:22:30.468627 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1362/2737] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:22:30 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:22:30.473320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d164093-560f-4df4-8048-43ce7e114adc None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:22:30.484581 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d164093-560f-4df4-8048-43ce7e114adc None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:22:30.543688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef30d6-3bcf-4003-b067-278d975abb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:22:30.544318 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef30d6-3bcf-4003-b067-278d975abb12 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:22:30.544590 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef30d6-3bcf-4003-b067-278d975abb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:22:30.544856 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef30d6-3bcf-4003-b067-278d975abb12 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:22:30.545500 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ef30d6-3bcf-4003-b067-278d975abb12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:22:30.547730 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ef30d6-3bcf-4003-b067-278d975abb12 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:22:30.570144 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1376/2738] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:22:30 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:22:30.574471 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce0fa43-bf6b-4cf0-9c9b-0b57feff9ef4 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:22:30.585239 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce0fa43-bf6b-4cf0-9c9b-0b57feff9ef4 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:22:30.590967 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b4aa6-7fa9-4f3b-9565-d94603ea61d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:22:30.591596 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b4aa6-7fa9-4f3b-9565-d94603ea61d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:22:30.591820 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b4aa6-7fa9-4f3b-9565-d94603ea61d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:22:30.592050 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b4aa6-7fa9-4f3b-9565-d94603ea61d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:22:30.592680 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9b4aa6-7fa9-4f3b-9565-d94603ea61d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:22:30.594319 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9b4aa6-7fa9-4f3b-9565-d94603ea61d5 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:22:30.604891 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1363/2739] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:22:30.609395 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef30d6-3bcf-4003-b067-278d975abb12 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:22:30.616002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:22:30.620154 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef30d6-3bcf-4003-b067-278d975abb12 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:22:30.626092 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383bce68-956c-4152-9e9e-fd5dc8f1f39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:22:30.626766 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bce68-956c-4152-9e9e-fd5dc8f1f39c admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:22:30.627037 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bce68-956c-4152-9e9e-fd5dc8f1f39c admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:22:30.627295 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383bce68-956c-4152-9e9e-fd5dc8f1f39c admin admin] PATH_INFO: `/v3/projects/d3e2495c0e4e4d7e957d8ddd76b4e01a/users/8ba5dfe97e1247fc8ec2d479cdac8c6a/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:22:30.644030 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383bce68-956c-4152-9e9e-fd5dc8f1f39c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3e2495c0e4e4d7e957d8ddd76b4e01a, user_id=8ba5dfe97e1247fc8ec2d479cdac8c6a, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:22:30.646729 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383bce68-956c-4152-9e9e-fd5dc8f1f39c admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:22:30.662601 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1377/2740] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:22:30 2022] PUT /identity/v3/projects/d3e2495c0e4e4d7e957d8ddd76b4e01a/users/8ba5dfe97e1247fc8ec2d479cdac8c6a/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:22:30.666855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b4aa6-7fa9-4f3b-9565-d94603ea61d5 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:22:30.677576 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b4aa6-7fa9-4f3b-9565-d94603ea61d5 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:22:30.743685 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a665eafb-446f-4f5c-9d87-f5a45490d89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:22:30.744285 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665eafb-446f-4f5c-9d87-f5a45490d89c admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:22:30.744570 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665eafb-446f-4f5c-9d87-f5a45490d89c admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:22:30.744833 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665eafb-446f-4f5c-9d87-f5a45490d89c admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:22:30.745501 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a665eafb-446f-4f5c-9d87-f5a45490d89c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:22:30.747111 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a665eafb-446f-4f5c-9d87-f5a45490d89c admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:22:30.758215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1364/2741] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:22:30.762567 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383bce68-956c-4152-9e9e-fd5dc8f1f39c admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:22:30.773656 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383bce68-956c-4152-9e9e-fd5dc8f1f39c admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:22:30.779483 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f365077-a126-4378-9abc-242d980688d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:22:30.780118 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f365077-a126-4378-9abc-242d980688d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:22:30.780384 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f365077-a126-4378-9abc-242d980688d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:22:30.780646 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f365077-a126-4378-9abc-242d980688d5 admin admin] PATH_INFO: `/v3/projects/d3e2495c0e4e4d7e957d8ddd76b4e01a/users/8ba5dfe97e1247fc8ec2d479cdac8c6a/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:22:30.783394 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f365077-a126-4378-9abc-242d980688d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3e2495c0e4e4d7e957d8ddd76b4e01a, user_id=8ba5dfe97e1247fc8ec2d479cdac8c6a, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:22:30.786029 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f365077-a126-4378-9abc-242d980688d5 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:22:30.797624 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1378/2742] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:22:30 2022] PUT /identity/v3/projects/d3e2495c0e4e4d7e957d8ddd76b4e01a/users/8ba5dfe97e1247fc8ec2d479cdac8c6a/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:22:30.810431 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d4a01-7989-4493-9444-453ba84ee6e5 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:22:30.810706 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d4a01-7989-4493-9444-453ba84ee6e5 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:22:30.810988 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d4a01-7989-4493-9444-453ba84ee6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:22:30.845731 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-e99d4a01-7989-4493-9444-453ba84ee6e5 None None] MFA Rules not processed for user `8ba5dfe97e1247fc8ec2d479cdac8c6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:22:30.847170 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-e99d4a01-7989-4493-9444-453ba84ee6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:22:30.929324 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1365/2743] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:22:30 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:22:30.943368 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f365077-a126-4378-9abc-242d980688d5 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:22:30.945162 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-6f365077-a126-4378-9abc-242d980688d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:22:31.011087 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f365077-a126-4378-9abc-242d980688d5 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:22:31.015077 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d32ae-fa3e-4efb-af97-35382bca593b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:22:31.015693 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d32ae-fa3e-4efb-af97-35382bca593b service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:22:31.015965 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d32ae-fa3e-4efb-af97-35382bca593b service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:22:31.016231 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d32ae-fa3e-4efb-af97-35382bca593b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:22:31.030207 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d32ae-fa3e-4efb-af97-35382bca593b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:22:31.033681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d32ae-fa3e-4efb-af97-35382bca593b service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:22:31.049737 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1379/2744] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:22:30 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:22:31.132712 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d4a01-7989-4493-9444-453ba84ee6e5 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:22:31.143478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d4a01-7989-4493-9444-453ba84ee6e5 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:22:31.147386 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42e65b-ee6f-4e22-a346-d131369d6f6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:22:31.148302 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42e65b-ee6f-4e22-a346-d131369d6f6a service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:22:31.148302 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42e65b-ee6f-4e22-a346-d131369d6f6a service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:22:31.148540 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42e65b-ee6f-4e22-a346-d131369d6f6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:22:31.160340 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:22:31.216909 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42e65b-ee6f-4e22-a346-d131369d6f6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:22:31.218310 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42e65b-ee6f-4e22-a346-d131369d6f6a service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:22:31.229213 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1366/2745] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:22:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:22:41.043555 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=39,keystone:INSERT=2 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:22:41.224727 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=22 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:10.602941 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d32ae-fa3e-4efb-af97-35382bca593b service cinder] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:10.668695 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d32ae-fa3e-4efb-af97-35382bca593b service cinder] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:10.673777 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee79cb8-8def-4b68-9644-a0e88db00c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:10.674395 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee79cb8-8def-4b68-9644-a0e88db00c4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:10.674677 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee79cb8-8def-4b68-9644-a0e88db00c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:10.674952 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee79cb8-8def-4b68-9644-a0e88db00c4f admin admin] PATH_INFO: `/v3/users/644f9e8b80a54fc0a970e3077b9b8155` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:10.676162 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee79cb8-8def-4b68-9644-a0e88db00c4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=644f9e8b80a54fc0a970e3077b9b8155)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:10.678124 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee79cb8-8def-4b68-9644-a0e88db00c4f admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:10.727132 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-6ee79cb8-8def-4b68-9644-a0e88db00c4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '644f9e8b80a54fc0a970e3077b9b8155'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:10.734545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-6ee79cb8-8def-4b68-9644-a0e88db00c4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '644f9e8b80a54fc0a970e3077b9b8155'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:10.746464 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-6ee79cb8-8def-4b68-9644-a0e88db00c4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '644f9e8b80a54fc0a970e3077b9b8155'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:10.747102 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-6ee79cb8-8def-4b68-9644-a0e88db00c4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '644f9e8b80a54fc0a970e3077b9b8155'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:10.754379 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-6ee79cb8-8def-4b68-9644-a0e88db00c4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '644f9e8b80a54fc0a970e3077b9b8155'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:10.760850 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1380/2746] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:23:10 2022] DELETE /identity/v3/users/644f9e8b80a54fc0a970e3077b9b8155 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:23:10.765857 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594979d-7565-439f-837a-d4a4a4370242 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:10.766369 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594979d-7565-439f-837a-d4a4a4370242 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:10.766369 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594979d-7565-439f-837a-d4a4a4370242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:10.791007 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-7594979d-7565-439f-837a-d4a4a4370242 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:23:10.792230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-7594979d-7565-439f-837a-d4a4a4370242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:10.861001 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1367/2747] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:23:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:23:10.874770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee79cb8-8def-4b68-9644-a0e88db00c4f admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:10.877305 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-6ee79cb8-8def-4b68-9644-a0e88db00c4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:10.942756 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee79cb8-8def-4b68-9644-a0e88db00c4f admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:10.946794 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f631a50-4fdb-4778-b036-1b93906dffd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:10.947397 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f631a50-4fdb-4778-b036-1b93906dffd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:10.947673 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f631a50-4fdb-4778-b036-1b93906dffd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:10.947941 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f631a50-4fdb-4778-b036-1b93906dffd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:10.957906 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f631a50-4fdb-4778-b036-1b93906dffd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:10.959388 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f631a50-4fdb-4778-b036-1b93906dffd5 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:10.970209 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1381/2748] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:23:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:23:11.255803 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7594979d-7565-439f-837a-d4a4a4370242 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:11.258368 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-7594979d-7565-439f-837a-d4a4a4370242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:11.276002 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7594979d-7565-439f-837a-d4a4a4370242 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:11.281016 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece38e3-2ce5-4d9f-9176-cd1f5e2e9fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:11.281760 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece38e3-2ce5-4d9f-9176-cd1f5e2e9fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:11.282031 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece38e3-2ce5-4d9f-9176-cd1f5e2e9fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:11.282377 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece38e3-2ce5-4d9f-9176-cd1f5e2e9fa1 admin admin] PATH_INFO: `/v3/projects/277522ac9b0e4dac8c312060a69ec295` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:11.283461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece38e3-2ce5-4d9f-9176-cd1f5e2e9fa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=277522ac9b0e4dac8c312060a69ec295)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:11.285399 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece38e3-2ce5-4d9f-9176-cd1f5e2e9fa1 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:11.332428 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cece38e3-2ce5-4d9f-9176-cd1f5e2e9fa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '277522ac9b0e4dac8c312060a69ec295', 'request_id': 'req-cece38e3-2ce5-4d9f-9176-cd1f5e2e9fa1'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:11.340821 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cece38e3-2ce5-4d9f-9176-cd1f5e2e9fa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '277522ac9b0e4dac8c312060a69ec295', 'request_id': 'req-cece38e3-2ce5-4d9f-9176-cd1f5e2e9fa1'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:11.346386 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cece38e3-2ce5-4d9f-9176-cd1f5e2e9fa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '277522ac9b0e4dac8c312060a69ec295', 'request_id': 'req-cece38e3-2ce5-4d9f-9176-cd1f5e2e9fa1'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:11.357473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cece38e3-2ce5-4d9f-9176-cd1f5e2e9fa1 admin admin] The token cache is being invalidate because project 277522ac9b0e4dac8c312060a69ec295 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:23:11.357752 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cece38e3-2ce5-4d9f-9176-cd1f5e2e9fa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:11.358390 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cece38e3-2ce5-4d9f-9176-cd1f5e2e9fa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:11.359844 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1368/2749] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:23:11 2022] DELETE /identity/v3/projects/277522ac9b0e4dac8c312060a69ec295 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:11.363904 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f631a50-4fdb-4778-b036-1b93906dffd5 service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:11.373783 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f631a50-4fdb-4778-b036-1b93906dffd5 service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:11.393262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:11.436007 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a380ab8-e518-4084-98b9-553fe2f81cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:11.436609 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a380ab8-e518-4084-98b9-553fe2f81cf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:11.436915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a380ab8-e518-4084-98b9-553fe2f81cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:11.437186 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a380ab8-e518-4084-98b9-553fe2f81cf2 admin admin] PATH_INFO: `/v3/users/8ba5dfe97e1247fc8ec2d479cdac8c6a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:11.438341 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a380ab8-e518-4084-98b9-553fe2f81cf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ba5dfe97e1247fc8ec2d479cdac8c6a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:11.440240 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a380ab8-e518-4084-98b9-553fe2f81cf2 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:11.500037 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5a380ab8-e518-4084-98b9-553fe2f81cf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ba5dfe97e1247fc8ec2d479cdac8c6a'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:11.509272 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5a380ab8-e518-4084-98b9-553fe2f81cf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ba5dfe97e1247fc8ec2d479cdac8c6a'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:11.521834 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5a380ab8-e518-4084-98b9-553fe2f81cf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ba5dfe97e1247fc8ec2d479cdac8c6a'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:11.522484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5a380ab8-e518-4084-98b9-553fe2f81cf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ba5dfe97e1247fc8ec2d479cdac8c6a'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:11.529718 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5a380ab8-e518-4084-98b9-553fe2f81cf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ba5dfe97e1247fc8ec2d479cdac8c6a'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:11.536395 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1382/2750] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:23:11 2022] DELETE /identity/v3/users/8ba5dfe97e1247fc8ec2d479cdac8c6a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:11.830151 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece38e3-2ce5-4d9f-9176-cd1f5e2e9fa1 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:11.831662 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-cece38e3-2ce5-4d9f-9176-cd1f5e2e9fa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:11.844700 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece38e3-2ce5-4d9f-9176-cd1f5e2e9fa1 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:11.901491 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2da63-091d-4eda-b007-fa2ff060876c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:11.902121 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2da63-091d-4eda-b007-fa2ff060876c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:11.902390 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2da63-091d-4eda-b007-fa2ff060876c admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:11.902656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2da63-091d-4eda-b007-fa2ff060876c admin admin] PATH_INFO: `/v3/projects/d3e2495c0e4e4d7e957d8ddd76b4e01a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:11.903823 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2da63-091d-4eda-b007-fa2ff060876c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3e2495c0e4e4d7e957d8ddd76b4e01a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:11.905720 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2da63-091d-4eda-b007-fa2ff060876c admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:11.938482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:11.955888 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-28c2da63-091d-4eda-b007-fa2ff060876c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3e2495c0e4e4d7e957d8ddd76b4e01a', 'request_id': 'req-28c2da63-091d-4eda-b007-fa2ff060876c'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:11.964120 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-28c2da63-091d-4eda-b007-fa2ff060876c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3e2495c0e4e4d7e957d8ddd76b4e01a', 'request_id': 'req-28c2da63-091d-4eda-b007-fa2ff060876c'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:11.969915 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-28c2da63-091d-4eda-b007-fa2ff060876c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3e2495c0e4e4d7e957d8ddd76b4e01a', 'request_id': 'req-28c2da63-091d-4eda-b007-fa2ff060876c'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:11.981386 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-28c2da63-091d-4eda-b007-fa2ff060876c admin admin] The token cache is being invalidate because project d3e2495c0e4e4d7e957d8ddd76b4e01a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:23:11.981666 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-28c2da63-091d-4eda-b007-fa2ff060876c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:11.982375 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-28c2da63-091d-4eda-b007-fa2ff060876c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:11.983945 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1369/2751] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:23:11 2022] DELETE /identity/v3/projects/d3e2495c0e4e4d7e957d8ddd76b4e01a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:11.988994 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b8840-b0ea-4201-a240-340ce60d2a88 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:11.989517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b8840-b0ea-4201-a240-340ce60d2a88 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:11.989517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b8840-b0ea-4201-a240-340ce60d2a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:12.010719 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-c88b8840-b0ea-4201-a240-340ce60d2a88 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:23:12.011923 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-c88b8840-b0ea-4201-a240-340ce60d2a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:12.083036 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1383/2752] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:23:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:23:12.091310 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a504f80-ea97-459d-8ad0-6044c64ad2c5 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:12.091622 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a504f80-ea97-459d-8ad0-6044c64ad2c5 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:12.091891 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a504f80-ea97-459d-8ad0-6044c64ad2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:12.113603 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-5a504f80-ea97-459d-8ad0-6044c64ad2c5 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:23:12.114824 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-5a504f80-ea97-459d-8ad0-6044c64ad2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:12.124733 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1370/2753] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:23:12.130432 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b8840-b0ea-4201-a240-340ce60d2a88 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:12.144251 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b8840-b0ea-4201-a240-340ce60d2a88 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:12.150184 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb18c870-009f-48f5-a936-2e8d95fa6aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:12.150787 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18c870-009f-48f5-a936-2e8d95fa6aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:12.151061 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18c870-009f-48f5-a936-2e8d95fa6aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:12.151329 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18c870-009f-48f5-a936-2e8d95fa6aa1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:12.151971 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb18c870-009f-48f5-a936-2e8d95fa6aa1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:23:12.152279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb18c870-009f-48f5-a936-2e8d95fa6aa1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:12.154033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb18c870-009f-48f5-a936-2e8d95fa6aa1 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:12.165484 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1384/2754] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:23:12.170023 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a504f80-ea97-459d-8ad0-6044c64ad2c5 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:12.179466 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a504f80-ea97-459d-8ad0-6044c64ad2c5 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:12.184250 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841f410-49ef-4779-bbf5-74af858d6036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:12.185290 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841f410-49ef-4779-bbf5-74af858d6036 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:12.185560 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841f410-49ef-4779-bbf5-74af858d6036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:12.185826 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841f410-49ef-4779-bbf5-74af858d6036 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:12.186460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a841f410-49ef-4779-bbf5-74af858d6036 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:12.188351 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a841f410-49ef-4779-bbf5-74af858d6036 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:12.220201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1371/2755] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:23:12 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:23:12.225539 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb18c870-009f-48f5-a936-2e8d95fa6aa1 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:12.236586 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb18c870-009f-48f5-a936-2e8d95fa6aa1 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:12.267624 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:12.303181 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d98f8e-f617-4202-b569-83a47e960181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:12.303794 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d98f8e-f617-4202-b569-83a47e960181 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:12.304218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d98f8e-f617-4202-b569-83a47e960181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:12.304330 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d98f8e-f617-4202-b569-83a47e960181 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:12.304948 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d98f8e-f617-4202-b569-83a47e960181 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:12.306919 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d98f8e-f617-4202-b569-83a47e960181 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:12.330119 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1385/2756] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:23:12 2022] POST /identity/v3/users => generated 291 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:23:12.334522 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841f410-49ef-4779-bbf5-74af858d6036 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:12.344704 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841f410-49ef-4779-bbf5-74af858d6036 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:12.350544 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead5d22-14f8-42e0-b0b2-9fb1244ef91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:12.351513 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead5d22-14f8-42e0-b0b2-9fb1244ef91c admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:12.351985 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead5d22-14f8-42e0-b0b2-9fb1244ef91c admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:12.352443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead5d22-14f8-42e0-b0b2-9fb1244ef91c admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:12.353607 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ead5d22-14f8-42e0-b0b2-9fb1244ef91c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:12.356740 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ead5d22-14f8-42e0-b0b2-9fb1244ef91c admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:12.367798 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1372/2757] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:23:12.372284 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d98f8e-f617-4202-b569-83a47e960181 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:12.383111 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d98f8e-f617-4202-b569-83a47e960181 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:12.388613 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc70c70-2082-4583-98ad-b9a28ff62978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:12.389247 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc70c70-2082-4583-98ad-b9a28ff62978 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:12.389529 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc70c70-2082-4583-98ad-b9a28ff62978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:12.389801 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc70c70-2082-4583-98ad-b9a28ff62978 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:12.390448 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc70c70-2082-4583-98ad-b9a28ff62978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:12.392077 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc70c70-2082-4583-98ad-b9a28ff62978 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:12.403494 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1386/2758] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:23:12.407965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead5d22-14f8-42e0-b0b2-9fb1244ef91c admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:12.418588 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead5d22-14f8-42e0-b0b2-9fb1244ef91c admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:12.423372 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f724302-3d48-4fa5-8e55-e4e322a420c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:12.424019 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f724302-3d48-4fa5-8e55-e4e322a420c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:12.424295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f724302-3d48-4fa5-8e55-e4e322a420c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:12.424568 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f724302-3d48-4fa5-8e55-e4e322a420c4 admin admin] PATH_INFO: `/v3/projects/cea6ecc01566421da7afe3eb01e31c55/users/a8ead961bf6e4ad19896b4a4b3e58c4c/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:12.439825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f724302-3d48-4fa5-8e55-e4e322a420c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cea6ecc01566421da7afe3eb01e31c55, user_id=a8ead961bf6e4ad19896b4a4b3e58c4c, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:12.442487 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f724302-3d48-4fa5-8e55-e4e322a420c4 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:12.458415 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1373/2759] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:23:12 2022] PUT /identity/v3/projects/cea6ecc01566421da7afe3eb01e31c55/users/a8ead961bf6e4ad19896b4a4b3e58c4c/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:23:12.470151 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14643b42-d583-46a3-8e81-dcb24b32823b None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:12.470435 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14643b42-d583-46a3-8e81-dcb24b32823b None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:12.470673 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14643b42-d583-46a3-8e81-dcb24b32823b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:12.500183 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-14643b42-d583-46a3-8e81-dcb24b32823b None None] MFA Rules not processed for user `a8ead961bf6e4ad19896b4a4b3e58c4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:23:12.501469 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-14643b42-d583-46a3-8e81-dcb24b32823b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:12.567538 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1387/2760] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:23:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:23:12.574058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f724302-3d48-4fa5-8e55-e4e322a420c4 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:12.635415 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f724302-3d48-4fa5-8e55-e4e322a420c4 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:12.640862 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ebf62-4c28-4197-a446-7c90877cbb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:12.641489 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ebf62-4c28-4197-a446-7c90877cbb0c admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:12.641747 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ebf62-4c28-4197-a446-7c90877cbb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:12.642180 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ebf62-4c28-4197-a446-7c90877cbb0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:12.642802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ebf62-4c28-4197-a446-7c90877cbb0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:12.644692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ebf62-4c28-4197-a446-7c90877cbb0c admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:12.677949 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1374/2761] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:23:12 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:23:12.683026 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14643b42-d583-46a3-8e81-dcb24b32823b None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:12.694874 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14643b42-d583-46a3-8e81-dcb24b32823b None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:12.753033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2c0ea-ca62-4a0c-b6a9-7054de00b0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:12.753680 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2c0ea-ca62-4a0c-b6a9-7054de00b0fe admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:12.753953 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2c0ea-ca62-4a0c-b6a9-7054de00b0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:12.754219 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2c0ea-ca62-4a0c-b6a9-7054de00b0fe admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:12.754838 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2c0ea-ca62-4a0c-b6a9-7054de00b0fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:12.756774 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2c0ea-ca62-4a0c-b6a9-7054de00b0fe admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:12.780914 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1388/2762] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:23:12 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:23:12.786409 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ebf62-4c28-4197-a446-7c90877cbb0c admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:12.796931 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ebf62-4c28-4197-a446-7c90877cbb0c admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:12.802544 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27565ced-a7fd-4922-be91-fdaf59498035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:12.803139 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27565ced-a7fd-4922-be91-fdaf59498035 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:12.803477 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27565ced-a7fd-4922-be91-fdaf59498035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:12.803742 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27565ced-a7fd-4922-be91-fdaf59498035 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:12.804378 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27565ced-a7fd-4922-be91-fdaf59498035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:12.806027 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27565ced-a7fd-4922-be91-fdaf59498035 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:12.819905 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1375/2763] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:23:12.825398 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2c0ea-ca62-4a0c-b6a9-7054de00b0fe admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:12.835706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2c0ea-ca62-4a0c-b6a9-7054de00b0fe admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:12.840669 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace488e2-aa0d-4819-b8b5-990e326bf497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:12.841349 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace488e2-aa0d-4819-b8b5-990e326bf497 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:12.841623 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace488e2-aa0d-4819-b8b5-990e326bf497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:12.841897 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace488e2-aa0d-4819-b8b5-990e326bf497 admin admin] PATH_INFO: `/v3/projects/1d2f1f52edab4c3ea85e8bfb3610dd19/users/44034c713ab548fc9135ea02a4156469/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:12.857275 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace488e2-aa0d-4819-b8b5-990e326bf497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d2f1f52edab4c3ea85e8bfb3610dd19, user_id=44034c713ab548fc9135ea02a4156469, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:12.859993 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace488e2-aa0d-4819-b8b5-990e326bf497 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:12.875377 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1389/2764] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:23:12 2022] PUT /identity/v3/projects/1d2f1f52edab4c3ea85e8bfb3610dd19/users/44034c713ab548fc9135ea02a4156469/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:12.879844 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27565ced-a7fd-4922-be91-fdaf59498035 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:12.890053 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27565ced-a7fd-4922-be91-fdaf59498035 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:12.910853 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:12.951326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27186b33-c111-4fa3-8134-04012ab05259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:12.952052 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27186b33-c111-4fa3-8134-04012ab05259 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:12.952327 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27186b33-c111-4fa3-8134-04012ab05259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:12.952592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27186b33-c111-4fa3-8134-04012ab05259 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:12.953307 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27186b33-c111-4fa3-8134-04012ab05259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:12.954845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27186b33-c111-4fa3-8134-04012ab05259 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:12.964338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1376/2765] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:23:12.968726 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace488e2-aa0d-4819-b8b5-990e326bf497 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:12.974315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:12.981031 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace488e2-aa0d-4819-b8b5-990e326bf497 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:12.986339 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a40ac73-43eb-48ca-a800-2d950c00d667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:12.986986 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40ac73-43eb-48ca-a800-2d950c00d667 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:12.987257 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40ac73-43eb-48ca-a800-2d950c00d667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:12.987521 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40ac73-43eb-48ca-a800-2d950c00d667 admin admin] PATH_INFO: `/v3/projects/1d2f1f52edab4c3ea85e8bfb3610dd19/users/44034c713ab548fc9135ea02a4156469/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:12.989600 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a40ac73-43eb-48ca-a800-2d950c00d667 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d2f1f52edab4c3ea85e8bfb3610dd19, user_id=44034c713ab548fc9135ea02a4156469, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:12.992265 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a40ac73-43eb-48ca-a800-2d950c00d667 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:13.003727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1390/2766] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:23:12 2022] PUT /identity/v3/projects/1d2f1f52edab4c3ea85e8bfb3610dd19/users/44034c713ab548fc9135ea02a4156469/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:13.014280 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5ffb0-8a43-481a-ab54-fe23c99b8fe1 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:13.014563 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5ffb0-8a43-481a-ab54-fe23c99b8fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:13.014834 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5ffb0-8a43-481a-ab54-fe23c99b8fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:13.046802 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-dff5ffb0-8a43-481a-ab54-fe23c99b8fe1 None None] MFA Rules not processed for user `44034c713ab548fc9135ea02a4156469`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:23:13.048020 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-dff5ffb0-8a43-481a-ab54-fe23c99b8fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:13.136987 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1377/2767] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:23:13 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:23:13.148809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a40ac73-43eb-48ca-a800-2d950c00d667 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:13.150681 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-5a40ac73-43eb-48ca-a800-2d950c00d667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:13.211261 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a40ac73-43eb-48ca-a800-2d950c00d667 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:13.215014 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7d8fe-d02f-4f71-ab38-0991e56c4d11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:13.215617 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7d8fe-d02f-4f71-ab38-0991e56c4d11 service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:13.215887 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7d8fe-d02f-4f71-ab38-0991e56c4d11 service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:13.216148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7d8fe-d02f-4f71-ab38-0991e56c4d11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:13.227126 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed7d8fe-d02f-4f71-ab38-0991e56c4d11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:13.230246 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed7d8fe-d02f-4f71-ab38-0991e56c4d11 service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:13.243765 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1391/2768] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:23:13 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:23:13.330175 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5ffb0-8a43-481a-ab54-fe23c99b8fe1 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:13.340623 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5ffb0-8a43-481a-ab54-fe23c99b8fe1 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:13.344744 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a18d3-56b5-4ec5-af52-c48f7fc22746 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:13.345397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a18d3-56b5-4ec5-af52-c48f7fc22746 service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:13.345672 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a18d3-56b5-4ec5-af52-c48f7fc22746 service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:13.345943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a18d3-56b5-4ec5-af52-c48f7fc22746 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:13.404804 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a18d3-56b5-4ec5-af52-c48f7fc22746 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:13.406196 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a18d3-56b5-4ec5-af52-c48f7fc22746 service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:13.415325 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1378/2769] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:23:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:23:22.468602 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7d8fe-d02f-4f71-ab38-0991e56c4d11 service cinder] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:22.539980 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7d8fe-d02f-4f71-ab38-0991e56c4d11 service cinder] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:22.544805 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761606b2-ba7d-4f24-aee3-cab147df40a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:22.545475 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761606b2-ba7d-4f24-aee3-cab147df40a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:22.545746 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761606b2-ba7d-4f24-aee3-cab147df40a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:22.546040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761606b2-ba7d-4f24-aee3-cab147df40a7 admin admin] PATH_INFO: `/v3/users/a8ead961bf6e4ad19896b4a4b3e58c4c` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:22.547173 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761606b2-ba7d-4f24-aee3-cab147df40a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8ead961bf6e4ad19896b4a4b3e58c4c)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:22.549070 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761606b2-ba7d-4f24-aee3-cab147df40a7 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:22.598824 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-761606b2-ba7d-4f24-aee3-cab147df40a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8ead961bf6e4ad19896b4a4b3e58c4c'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:22.606924 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-761606b2-ba7d-4f24-aee3-cab147df40a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8ead961bf6e4ad19896b4a4b3e58c4c'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:22.619373 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-761606b2-ba7d-4f24-aee3-cab147df40a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8ead961bf6e4ad19896b4a4b3e58c4c'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:22.620091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-761606b2-ba7d-4f24-aee3-cab147df40a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8ead961bf6e4ad19896b4a4b3e58c4c'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:22.627915 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-761606b2-ba7d-4f24-aee3-cab147df40a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8ead961bf6e4ad19896b4a4b3e58c4c'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:22.634430 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1392/2770] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:23:22 2022] DELETE /identity/v3/users/a8ead961bf6e4ad19896b4a4b3e58c4c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:22.639487 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8069a-1276-40fb-822f-15f10f3f1c26 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:22.640010 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8069a-1276-40fb-822f-15f10f3f1c26 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:22.640010 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8069a-1276-40fb-822f-15f10f3f1c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:22.659886 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-04c8069a-1276-40fb-822f-15f10f3f1c26 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:23:22.661082 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-04c8069a-1276-40fb-822f-15f10f3f1c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:22.701716 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:22.725876 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1379/2771] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:23:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:23:22.738307 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761606b2-ba7d-4f24-aee3-cab147df40a7 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:22.739927 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-761606b2-ba7d-4f24-aee3-cab147df40a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:22.756259 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:22.804604 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761606b2-ba7d-4f24-aee3-cab147df40a7 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:22.808816 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba2aef8-49b1-4e09-bb1c-10e4487049e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:22.809457 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2aef8-49b1-4e09-bb1c-10e4487049e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:22.810089 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2aef8-49b1-4e09-bb1c-10e4487049e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:22.810373 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2aef8-49b1-4e09-bb1c-10e4487049e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:22.820517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba2aef8-49b1-4e09-bb1c-10e4487049e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:22.822022 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba2aef8-49b1-4e09-bb1c-10e4487049e9 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:22.831543 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1393/2772] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:23:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:23:23.113891 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8069a-1276-40fb-822f-15f10f3f1c26 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:23.115427 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-04c8069a-1276-40fb-822f-15f10f3f1c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:23.128230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8069a-1276-40fb-822f-15f10f3f1c26 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:23.135659 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eae80b-d6c4-4584-ad77-ca88a15a2ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:23.136592 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eae80b-d6c4-4584-ad77-ca88a15a2ecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:23.137180 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eae80b-d6c4-4584-ad77-ca88a15a2ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:23.137498 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eae80b-d6c4-4584-ad77-ca88a15a2ecb admin admin] PATH_INFO: `/v3/projects/cea6ecc01566421da7afe3eb01e31c55` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:23.138711 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eae80b-d6c4-4584-ad77-ca88a15a2ecb admin admin] RBAC: Authorizing `identity:delete_project(project_id=cea6ecc01566421da7afe3eb01e31c55)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:23.141951 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eae80b-d6c4-4584-ad77-ca88a15a2ecb admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:23.192421 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-32eae80b-d6c4-4584-ad77-ca88a15a2ecb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cea6ecc01566421da7afe3eb01e31c55', 'request_id': 'req-32eae80b-d6c4-4584-ad77-ca88a15a2ecb'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:23.201215 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-32eae80b-d6c4-4584-ad77-ca88a15a2ecb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cea6ecc01566421da7afe3eb01e31c55', 'request_id': 'req-32eae80b-d6c4-4584-ad77-ca88a15a2ecb'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:23.207793 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-32eae80b-d6c4-4584-ad77-ca88a15a2ecb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cea6ecc01566421da7afe3eb01e31c55', 'request_id': 'req-32eae80b-d6c4-4584-ad77-ca88a15a2ecb'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:23.220159 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-32eae80b-d6c4-4584-ad77-ca88a15a2ecb admin admin] The token cache is being invalidate because project cea6ecc01566421da7afe3eb01e31c55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:23:23.220447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-32eae80b-d6c4-4584-ad77-ca88a15a2ecb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:23.221241 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-32eae80b-d6c4-4584-ad77-ca88a15a2ecb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:23.222747 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1380/2773] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:23:23 2022] DELETE /identity/v3/projects/cea6ecc01566421da7afe3eb01e31c55 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:23.230432 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba2aef8-49b1-4e09-bb1c-10e4487049e9 service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:23.240198 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba2aef8-49b1-4e09-bb1c-10e4487049e9 service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:23.304573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b6c79-a90d-4e22-88cb-07aa0caa5722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:23.305381 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b6c79-a90d-4e22-88cb-07aa0caa5722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:23.305659 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b6c79-a90d-4e22-88cb-07aa0caa5722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:23.305953 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b6c79-a90d-4e22-88cb-07aa0caa5722 admin admin] PATH_INFO: `/v3/users/44034c713ab548fc9135ea02a4156469` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:23.307111 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843b6c79-a90d-4e22-88cb-07aa0caa5722 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44034c713ab548fc9135ea02a4156469)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:23.309048 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843b6c79-a90d-4e22-88cb-07aa0caa5722 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:23.358436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-843b6c79-a90d-4e22-88cb-07aa0caa5722 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44034c713ab548fc9135ea02a4156469'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:23.366986 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-843b6c79-a90d-4e22-88cb-07aa0caa5722 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44034c713ab548fc9135ea02a4156469'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:23.379460 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-843b6c79-a90d-4e22-88cb-07aa0caa5722 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44034c713ab548fc9135ea02a4156469'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:23.380185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-843b6c79-a90d-4e22-88cb-07aa0caa5722 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44034c713ab548fc9135ea02a4156469'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:23.387370 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-843b6c79-a90d-4e22-88cb-07aa0caa5722 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44034c713ab548fc9135ea02a4156469'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:23.393125 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1394/2774] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:23:23 2022] DELETE /identity/v3/users/44034c713ab548fc9135ea02a4156469 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:23.684110 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eae80b-d6c4-4584-ad77-ca88a15a2ecb admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:23.686316 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-32eae80b-d6c4-4584-ad77-ca88a15a2ecb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:23.699518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eae80b-d6c4-4584-ad77-ca88a15a2ecb admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:23.755778 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41261841-3479-4736-a68d-bec71416b112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:23.756392 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41261841-3479-4736-a68d-bec71416b112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:23.756685 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41261841-3479-4736-a68d-bec71416b112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:23.756949 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41261841-3479-4736-a68d-bec71416b112 admin admin] PATH_INFO: `/v3/projects/1d2f1f52edab4c3ea85e8bfb3610dd19` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:23.758178 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41261841-3479-4736-a68d-bec71416b112 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d2f1f52edab4c3ea85e8bfb3610dd19)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:23.760115 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41261841-3479-4736-a68d-bec71416b112 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:23.807466 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-41261841-3479-4736-a68d-bec71416b112 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d2f1f52edab4c3ea85e8bfb3610dd19', 'request_id': 'req-41261841-3479-4736-a68d-bec71416b112'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:23.815581 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-41261841-3479-4736-a68d-bec71416b112 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d2f1f52edab4c3ea85e8bfb3610dd19', 'request_id': 'req-41261841-3479-4736-a68d-bec71416b112'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:23.821308 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-41261841-3479-4736-a68d-bec71416b112 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d2f1f52edab4c3ea85e8bfb3610dd19', 'request_id': 'req-41261841-3479-4736-a68d-bec71416b112'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:23.828160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:23.833446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-41261841-3479-4736-a68d-bec71416b112 admin admin] The token cache is being invalidate because project 1d2f1f52edab4c3ea85e8bfb3610dd19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:23:23.833766 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-41261841-3479-4736-a68d-bec71416b112 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:23.834523 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-41261841-3479-4736-a68d-bec71416b112 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:23.840125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1381/2775] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:23:23 2022] DELETE /identity/v3/projects/1d2f1f52edab4c3ea85e8bfb3610dd19 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:23.841760 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed612e-7f3a-4155-892d-4f668ba49417 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:23.842283 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed612e-7f3a-4155-892d-4f668ba49417 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:23.842283 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed612e-7f3a-4155-892d-4f668ba49417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:23.863807 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-03ed612e-7f3a-4155-892d-4f668ba49417 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:23:23.865031 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-03ed612e-7f3a-4155-892d-4f668ba49417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:23.896564 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:23.933106 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1395/2776] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:23:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:23:23.942961 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ce195-4fb0-40b1-bb76-79b59e5df383 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:23.943204 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ce195-4fb0-40b1-bb76-79b59e5df383 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:23.943504 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ce195-4fb0-40b1-bb76-79b59e5df383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:23.964337 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-b18ce195-4fb0-40b1-bb76-79b59e5df383 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:23:23.965609 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b18ce195-4fb0-40b1-bb76-79b59e5df383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:23.974006 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1382/2777] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:23:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:23:24.037860 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed612e-7f3a-4155-892d-4f668ba49417 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:24.048067 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed612e-7f3a-4155-892d-4f668ba49417 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:24.053055 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7cc65-daa1-460a-93ad-9f991d06e08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:24.053706 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7cc65-daa1-460a-93ad-9f991d06e08a admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:24.053979 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7cc65-daa1-460a-93ad-9f991d06e08a admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:24.054245 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7cc65-daa1-460a-93ad-9f991d06e08a admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:24.054930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee7cc65-daa1-460a-93ad-9f991d06e08a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:23:24.055241 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee7cc65-daa1-460a-93ad-9f991d06e08a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:24.057009 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee7cc65-daa1-460a-93ad-9f991d06e08a admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:24.069812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1396/2778] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 15:23:24.074713 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ce195-4fb0-40b1-bb76-79b59e5df383 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:24.084261 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ce195-4fb0-40b1-bb76-79b59e5df383 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:24.090151 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f4491-cd96-4115-b7ae-ff2e2174c8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:24.090807 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f4491-cd96-4115-b7ae-ff2e2174c8ae admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:24.091082 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f4491-cd96-4115-b7ae-ff2e2174c8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:24.091350 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f4491-cd96-4115-b7ae-ff2e2174c8ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:24.091992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f4491-cd96-4115-b7ae-ff2e2174c8ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:24.094029 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f4491-cd96-4115-b7ae-ff2e2174c8ae admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:24.124725 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1383/2779] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:23:24 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:23:24.129239 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7cc65-daa1-460a-93ad-9f991d06e08a admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:24.139820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7cc65-daa1-460a-93ad-9f991d06e08a admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:24.202961 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9f484-1ec1-441d-862f-7fa19df8cc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:24.203450 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9f484-1ec1-441d-862f-7fa19df8cc1b admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:24.203727 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9f484-1ec1-441d-862f-7fa19df8cc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:24.203982 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9f484-1ec1-441d-862f-7fa19df8cc1b admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:24.204624 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc9f484-1ec1-441d-862f-7fa19df8cc1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:24.206797 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc9f484-1ec1-441d-862f-7fa19df8cc1b admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:24.234806 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1397/2780] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:23:24 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:23:24.239403 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f4491-cd96-4115-b7ae-ff2e2174c8ae admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:24.251181 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f4491-cd96-4115-b7ae-ff2e2174c8ae admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:24.256581 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb43941-519d-4d13-8b6a-43634b004a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:24.257236 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb43941-519d-4d13-8b6a-43634b004a1a admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:24.257518 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb43941-519d-4d13-8b6a-43634b004a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:24.257781 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb43941-519d-4d13-8b6a-43634b004a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:24.258529 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb43941-519d-4d13-8b6a-43634b004a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:24.260172 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb43941-519d-4d13-8b6a-43634b004a1a admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:24.271440 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1384/2781] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:23:24.275825 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9f484-1ec1-441d-862f-7fa19df8cc1b admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:24.287323 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9f484-1ec1-441d-862f-7fa19df8cc1b admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:24.292323 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3025cbd0-792a-4650-aeff-6f9048f2bced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:24.292959 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025cbd0-792a-4650-aeff-6f9048f2bced admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:24.293274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025cbd0-792a-4650-aeff-6f9048f2bced admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:24.293542 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025cbd0-792a-4650-aeff-6f9048f2bced admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:24.294190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3025cbd0-792a-4650-aeff-6f9048f2bced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:24.295851 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3025cbd0-792a-4650-aeff-6f9048f2bced admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:24.306893 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1398/2782] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:23:24.312978 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb43941-519d-4d13-8b6a-43634b004a1a admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:24.323793 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb43941-519d-4d13-8b6a-43634b004a1a admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:24.329246 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ffcfa5-7449-43e3-b768-7c1c99c66ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:24.329919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ffcfa5-7449-43e3-b768-7c1c99c66ce5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:24.330190 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ffcfa5-7449-43e3-b768-7c1c99c66ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:24.330484 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ffcfa5-7449-43e3-b768-7c1c99c66ce5 admin admin] PATH_INFO: `/v3/projects/2b3907d206d84b2a944438c233d3d567/users/22d71477a4c340548b1b09bc40255bcb/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:24.347670 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ffcfa5-7449-43e3-b768-7c1c99c66ce5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b3907d206d84b2a944438c233d3d567, user_id=22d71477a4c340548b1b09bc40255bcb, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:24.350366 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ffcfa5-7449-43e3-b768-7c1c99c66ce5 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:24.369449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1385/2783] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:23:24 2022] PUT /identity/v3/projects/2b3907d206d84b2a944438c233d3d567/users/22d71477a4c340548b1b09bc40255bcb/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:24.380821 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f99dc4-243c-44c5-8864-6b6b844d9b72 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:24.381261 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f99dc4-243c-44c5-8864-6b6b844d9b72 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:24.381594 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f99dc4-243c-44c5-8864-6b6b844d9b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:24.410648 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-18f99dc4-243c-44c5-8864-6b6b844d9b72 None None] MFA Rules not processed for user `22d71477a4c340548b1b09bc40255bcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:23:24.411908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-18f99dc4-243c-44c5-8864-6b6b844d9b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:24.503397 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1399/2784] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:23:24 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:23:24.509675 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ffcfa5-7449-43e3-b768-7c1c99c66ce5 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:24.576995 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ffcfa5-7449-43e3-b768-7c1c99c66ce5 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:24.582430 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af18a1c4-0feb-4616-a51d-445a26fe98aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:24.583029 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18a1c4-0feb-4616-a51d-445a26fe98aa admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:24.583304 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18a1c4-0feb-4616-a51d-445a26fe98aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:24.583570 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18a1c4-0feb-4616-a51d-445a26fe98aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:24.584194 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af18a1c4-0feb-4616-a51d-445a26fe98aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:24.586268 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af18a1c4-0feb-4616-a51d-445a26fe98aa admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:24.619372 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1386/2785] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:23:24 2022] POST /identity/v3/projects => generated 387 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:23:24.623834 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f99dc4-243c-44c5-8864-6b6b844d9b72 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:24.635144 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f99dc4-243c-44c5-8864-6b6b844d9b72 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:24.662564 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:24.696115 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc1fb9-713b-4232-8349-d77e823cd53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:24.696729 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc1fb9-713b-4232-8349-d77e823cd53b admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:24.697004 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc1fb9-713b-4232-8349-d77e823cd53b admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:24.697309 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc1fb9-713b-4232-8349-d77e823cd53b admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:24.697968 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc1fb9-713b-4232-8349-d77e823cd53b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:24.699875 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc1fb9-713b-4232-8349-d77e823cd53b admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:24.728336 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1400/2786] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:23:24 2022] POST /identity/v3/users => generated 300 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:23:24.732713 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af18a1c4-0feb-4616-a51d-445a26fe98aa admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:24.743419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af18a1c4-0feb-4616-a51d-445a26fe98aa admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:24.748076 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0dbfc-e130-414d-ad6f-69d1221d836d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:24.748665 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0dbfc-e130-414d-ad6f-69d1221d836d admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:24.748940 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0dbfc-e130-414d-ad6f-69d1221d836d admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:24.749236 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0dbfc-e130-414d-ad6f-69d1221d836d admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:24.749847 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d0dbfc-e130-414d-ad6f-69d1221d836d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:24.751459 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d0dbfc-e130-414d-ad6f-69d1221d836d admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:24.762630 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1387/2787] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:23:24.767001 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc1fb9-713b-4232-8349-d77e823cd53b admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:24.777931 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc1fb9-713b-4232-8349-d77e823cd53b admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:24.783218 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48760da-405b-4070-bf02-cd07b1ff22f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:24.783867 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48760da-405b-4070-bf02-cd07b1ff22f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:24.784145 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48760da-405b-4070-bf02-cd07b1ff22f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:24.784409 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48760da-405b-4070-bf02-cd07b1ff22f1 admin admin] PATH_INFO: `/v3/projects/3a430a9d5a7441eeb206d18346c35ee2/users/e2acb5fd03584976b8925a65c3276d51/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:24.801069 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48760da-405b-4070-bf02-cd07b1ff22f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a430a9d5a7441eeb206d18346c35ee2, user_id=e2acb5fd03584976b8925a65c3276d51, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:24.803844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48760da-405b-4070-bf02-cd07b1ff22f1 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:24.820741 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1401/2788] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:23:24 2022] PUT /identity/v3/projects/3a430a9d5a7441eeb206d18346c35ee2/users/e2acb5fd03584976b8925a65c3276d51/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:23:24.824833 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0dbfc-e130-414d-ad6f-69d1221d836d admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:24.835338 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0dbfc-e130-414d-ad6f-69d1221d836d admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:24.893295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f5cf11-327b-4889-abd2-43e80215de7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:24.893897 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5cf11-327b-4889-abd2-43e80215de7e admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:24.894169 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5cf11-327b-4889-abd2-43e80215de7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:24.894451 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5cf11-327b-4889-abd2-43e80215de7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:24.895186 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f5cf11-327b-4889-abd2-43e80215de7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:24.896841 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f5cf11-327b-4889-abd2-43e80215de7e admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:24.906816 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1388/2789] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:23:24.913494 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48760da-405b-4070-bf02-cd07b1ff22f1 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:24.925272 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48760da-405b-4070-bf02-cd07b1ff22f1 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:24.930908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf8fc0-2104-4243-9549-657aad14ecb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:24.931568 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf8fc0-2104-4243-9549-657aad14ecb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:24.931842 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf8fc0-2104-4243-9549-657aad14ecb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:24.932108 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf8fc0-2104-4243-9549-657aad14ecb2 admin admin] PATH_INFO: `/v3/projects/3a430a9d5a7441eeb206d18346c35ee2/users/e2acb5fd03584976b8925a65c3276d51/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:24.934296 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcf8fc0-2104-4243-9549-657aad14ecb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a430a9d5a7441eeb206d18346c35ee2, user_id=e2acb5fd03584976b8925a65c3276d51, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:24.937315 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcf8fc0-2104-4243-9549-657aad14ecb2 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:24.948962 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1402/2790] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:23:24 2022] PUT /identity/v3/projects/3a430a9d5a7441eeb206d18346c35ee2/users/e2acb5fd03584976b8925a65c3276d51/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:24.964849 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5e9e8-fa84-4459-9f82-3d322ad3e83a None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:24.965344 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5e9e8-fa84-4459-9f82-3d322ad3e83a None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:24.965738 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5e9e8-fa84-4459-9f82-3d322ad3e83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:24.979278 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:25.001595 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-61f5e9e8-fa84-4459-9f82-3d322ad3e83a None None] MFA Rules not processed for user `e2acb5fd03584976b8925a65c3276d51`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:23:25.002859 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-61f5e9e8-fa84-4459-9f82-3d322ad3e83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:25.073089 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1389/2791] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:23:24 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:23:25.084663 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf8fc0-2104-4243-9549-657aad14ecb2 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:25.086590 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-fbcf8fc0-2104-4243-9549-657aad14ecb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:25.153422 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf8fc0-2104-4243-9549-657aad14ecb2 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:25.158082 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d6987-cb04-4d46-b3c0-b9fc91981fee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:25.158720 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d6987-cb04-4d46-b3c0-b9fc91981fee service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:25.158992 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d6987-cb04-4d46-b3c0-b9fc91981fee service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:25.159262 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d6987-cb04-4d46-b3c0-b9fc91981fee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:25.171310 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1d6987-cb04-4d46-b3c0-b9fc91981fee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:25.172784 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1d6987-cb04-4d46-b3c0-b9fc91981fee service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:25.185318 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1403/2792] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:23:25 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:23:25.218458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5e9e8-fa84-4459-9f82-3d322ad3e83a None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:25.288388 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5e9e8-fa84-4459-9f82-3d322ad3e83a None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:25.293410 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc974ea-0384-4ea9-b591-c1fd9fc755e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:25.294036 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc974ea-0384-4ea9-b591-c1fd9fc755e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:25.294314 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc974ea-0384-4ea9-b591-c1fd9fc755e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:25.294579 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc974ea-0384-4ea9-b591-c1fd9fc755e0 admin admin] PATH_INFO: `/v3/users/22d71477a4c340548b1b09bc40255bcb` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:25.295771 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc974ea-0384-4ea9-b591-c1fd9fc755e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22d71477a4c340548b1b09bc40255bcb)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:25.297757 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc974ea-0384-4ea9-b591-c1fd9fc755e0 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:25.348468 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-fdc974ea-0384-4ea9-b591-c1fd9fc755e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22d71477a4c340548b1b09bc40255bcb'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:25.356894 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-fdc974ea-0384-4ea9-b591-c1fd9fc755e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22d71477a4c340548b1b09bc40255bcb'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:25.369846 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-fdc974ea-0384-4ea9-b591-c1fd9fc755e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22d71477a4c340548b1b09bc40255bcb'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:25.370440 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-fdc974ea-0384-4ea9-b591-c1fd9fc755e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22d71477a4c340548b1b09bc40255bcb'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:25.377679 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-fdc974ea-0384-4ea9-b591-c1fd9fc755e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22d71477a4c340548b1b09bc40255bcb'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:25.383469 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1390/2793] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:23:25 2022] DELETE /identity/v3/users/22d71477a4c340548b1b09bc40255bcb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:25.388108 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a6ade-dc76-430c-861a-db1318aa0ecd None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:25.388387 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a6ade-dc76-430c-861a-db1318aa0ecd None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:25.388651 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a6ade-dc76-430c-861a-db1318aa0ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:25.408883 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-be7a6ade-dc76-430c-861a-db1318aa0ecd None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:23:25.410159 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-be7a6ade-dc76-430c-861a-db1318aa0ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:25.475362 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1404/2794] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:23:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:23:25.485657 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc974ea-0384-4ea9-b591-c1fd9fc755e0 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:25.487124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-fdc974ea-0384-4ea9-b591-c1fd9fc755e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:25.498742 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:25.548830 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc974ea-0384-4ea9-b591-c1fd9fc755e0 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:25.552653 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6afca-553e-4936-bed8-febec6257cd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:25.553298 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6afca-553e-4936-bed8-febec6257cd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:25.553578 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6afca-553e-4936-bed8-febec6257cd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:25.553834 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6afca-553e-4936-bed8-febec6257cd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:25.563900 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6afca-553e-4936-bed8-febec6257cd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:25.565372 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6afca-553e-4936-bed8-febec6257cd5 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:25.575816 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1391/2795] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:23:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:23:25.872587 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a6ade-dc76-430c-861a-db1318aa0ecd None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:25.874107 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-be7a6ade-dc76-430c-861a-db1318aa0ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:25.886623 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a6ade-dc76-430c-861a-db1318aa0ecd None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:25.892144 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3ff82-03aa-425b-88ff-712925ab59b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:25.892815 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3ff82-03aa-425b-88ff-712925ab59b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:25.893089 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3ff82-03aa-425b-88ff-712925ab59b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:25.893400 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3ff82-03aa-425b-88ff-712925ab59b1 admin admin] PATH_INFO: `/v3/projects/2b3907d206d84b2a944438c233d3d567` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:25.894683 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3ff82-03aa-425b-88ff-712925ab59b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b3907d206d84b2a944438c233d3d567)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:25.897547 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3ff82-03aa-425b-88ff-712925ab59b1 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:25.921678 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:25.948936 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-9ee3ff82-03aa-425b-88ff-712925ab59b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b3907d206d84b2a944438c233d3d567', 'request_id': 'req-9ee3ff82-03aa-425b-88ff-712925ab59b1'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:25.954617 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-9ee3ff82-03aa-425b-88ff-712925ab59b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b3907d206d84b2a944438c233d3d567', 'request_id': 'req-9ee3ff82-03aa-425b-88ff-712925ab59b1'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:25.961221 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-9ee3ff82-03aa-425b-88ff-712925ab59b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b3907d206d84b2a944438c233d3d567', 'request_id': 'req-9ee3ff82-03aa-425b-88ff-712925ab59b1'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:25.976194 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-9ee3ff82-03aa-425b-88ff-712925ab59b1 admin admin] The token cache is being invalidate because project 2b3907d206d84b2a944438c233d3d567 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:23:25.976481 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-9ee3ff82-03aa-425b-88ff-712925ab59b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:25.977163 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-9ee3ff82-03aa-425b-88ff-712925ab59b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:25.978659 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1405/2796] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:23:25 2022] DELETE /identity/v3/projects/2b3907d206d84b2a944438c233d3d567 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:25.983302 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6afca-553e-4936-bed8-febec6257cd5 service neutron] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:25.993671 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6afca-553e-4936-bed8-febec6257cd5 service neutron] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:26.054085 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3544e-2b44-45f5-92bf-621400120640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:26.054754 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3544e-2b44-45f5-92bf-621400120640 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:26.055055 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3544e-2b44-45f5-92bf-621400120640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:26.055311 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3544e-2b44-45f5-92bf-621400120640 admin admin] PATH_INFO: `/v3/users/e2acb5fd03584976b8925a65c3276d51` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:26.056505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e3544e-2b44-45f5-92bf-621400120640 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2acb5fd03584976b8925a65c3276d51)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:26.058587 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e3544e-2b44-45f5-92bf-621400120640 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:26.110755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-13e3544e-2b44-45f5-92bf-621400120640 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2acb5fd03584976b8925a65c3276d51'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:26.128329 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-13e3544e-2b44-45f5-92bf-621400120640 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2acb5fd03584976b8925a65c3276d51'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:26.144949 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-13e3544e-2b44-45f5-92bf-621400120640 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2acb5fd03584976b8925a65c3276d51'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:26.146088 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-13e3544e-2b44-45f5-92bf-621400120640 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2acb5fd03584976b8925a65c3276d51'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:26.153964 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-13e3544e-2b44-45f5-92bf-621400120640 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2acb5fd03584976b8925a65c3276d51'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:26.160154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1392/2797] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:23:25 2022] DELETE /identity/v3/users/e2acb5fd03584976b8925a65c3276d51 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:26.457056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3ff82-03aa-425b-88ff-712925ab59b1 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:26.458831 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-9ee3ff82-03aa-425b-88ff-712925ab59b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:26.473601 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3ff82-03aa-425b-88ff-712925ab59b1 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:26.544514 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e1efdc-1b76-4171-964a-81c151f9ab49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:26.545055 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1efdc-1b76-4171-964a-81c151f9ab49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:26.545356 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1efdc-1b76-4171-964a-81c151f9ab49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:26.545621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1efdc-1b76-4171-964a-81c151f9ab49 admin admin] PATH_INFO: `/v3/projects/3a430a9d5a7441eeb206d18346c35ee2` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:26.546833 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e1efdc-1b76-4171-964a-81c151f9ab49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a430a9d5a7441eeb206d18346c35ee2)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:26.548767 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e1efdc-1b76-4171-964a-81c151f9ab49 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:26.598719 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-00e1efdc-1b76-4171-964a-81c151f9ab49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a430a9d5a7441eeb206d18346c35ee2', 'request_id': 'req-00e1efdc-1b76-4171-964a-81c151f9ab49'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:26.609885 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-00e1efdc-1b76-4171-964a-81c151f9ab49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a430a9d5a7441eeb206d18346c35ee2', 'request_id': 'req-00e1efdc-1b76-4171-964a-81c151f9ab49'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:26.617122 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-00e1efdc-1b76-4171-964a-81c151f9ab49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a430a9d5a7441eeb206d18346c35ee2', 'request_id': 'req-00e1efdc-1b76-4171-964a-81c151f9ab49'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:26.630696 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-00e1efdc-1b76-4171-964a-81c151f9ab49 admin admin] The token cache is being invalidate because project 3a430a9d5a7441eeb206d18346c35ee2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:23:26.630981 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-00e1efdc-1b76-4171-964a-81c151f9ab49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:26.631625 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-00e1efdc-1b76-4171-964a-81c151f9ab49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:26.633110 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1406/2798] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:23:26 2022] DELETE /identity/v3/projects/3a430a9d5a7441eeb206d18346c35ee2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:26.638286 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd228a3c-bd58-4d91-a28b-d1b93da9b564 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:26.638544 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd228a3c-bd58-4d91-a28b-d1b93da9b564 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:26.638818 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd228a3c-bd58-4d91-a28b-d1b93da9b564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:26.660755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-bd228a3c-bd58-4d91-a28b-d1b93da9b564 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:23:26.662619 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-bd228a3c-bd58-4d91-a28b-d1b93da9b564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:26.690495 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:26.730056 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1393/2799] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:23:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:23:26.739809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da4634-5047-4971-a7aa-369f0309d2a4 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:26.740086 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da4634-5047-4971-a7aa-369f0309d2a4 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:26.740352 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da4634-5047-4971-a7aa-369f0309d2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:26.760971 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-07da4634-5047-4971-a7aa-369f0309d2a4 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:23:26.762205 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-07da4634-5047-4971-a7aa-369f0309d2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:26.770317 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1407/2800] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:23:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:23:26.774869 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd228a3c-bd58-4d91-a28b-d1b93da9b564 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:26.785541 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd228a3c-bd58-4d91-a28b-d1b93da9b564 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:26.791157 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0416c-cd62-4e92-9c16-e2160f0acc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:26.791764 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0416c-cd62-4e92-9c16-e2160f0acc86 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:26.792033 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0416c-cd62-4e92-9c16-e2160f0acc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:26.792283 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0416c-cd62-4e92-9c16-e2160f0acc86 admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:26.792925 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0416c-cd62-4e92-9c16-e2160f0acc86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:23:26.793264 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0416c-cd62-4e92-9c16-e2160f0acc86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:26.794947 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0416c-cd62-4e92-9c16-e2160f0acc86 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:26.807237 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1394/2801] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:23:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:23:26.812237 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da4634-5047-4971-a7aa-369f0309d2a4 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:26.822227 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da4634-5047-4971-a7aa-369f0309d2a4 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:26.828042 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf55336-ad6b-49b4-aa49-11ab196699c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:26.828665 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf55336-ad6b-49b4-aa49-11ab196699c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:26.828942 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf55336-ad6b-49b4-aa49-11ab196699c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:26.829253 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf55336-ad6b-49b4-aa49-11ab196699c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:26.829895 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf55336-ad6b-49b4-aa49-11ab196699c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:26.831827 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf55336-ad6b-49b4-aa49-11ab196699c3 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:26.862263 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1408/2802] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:23:26 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:23:26.866417 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0416c-cd62-4e92-9c16-e2160f0acc86 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:26.877562 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0416c-cd62-4e92-9c16-e2160f0acc86 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:26.947384 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930e5d5-127b-43b0-9a80-922db18226e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:26.948032 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930e5d5-127b-43b0-9a80-922db18226e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:26.948280 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930e5d5-127b-43b0-9a80-922db18226e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:26.948552 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930e5d5-127b-43b0-9a80-922db18226e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:26.949220 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9930e5d5-127b-43b0-9a80-922db18226e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:26.951209 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9930e5d5-127b-43b0-9a80-922db18226e0 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:26.976138 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1395/2803] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:23:26 2022] POST /identity/v3/users => generated 290 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:23:26.980576 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf55336-ad6b-49b4-aa49-11ab196699c3 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:26.991844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf55336-ad6b-49b4-aa49-11ab196699c3 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:26.997825 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb719afe-d2f1-4e81-acbc-a717814d213a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:26.998447 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb719afe-d2f1-4e81-acbc-a717814d213a admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:26.998714 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb719afe-d2f1-4e81-acbc-a717814d213a admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:26.998975 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb719afe-d2f1-4e81-acbc-a717814d213a admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:26.999654 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb719afe-d2f1-4e81-acbc-a717814d213a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:27.001327 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb719afe-d2f1-4e81-acbc-a717814d213a admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:27.004465 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:27.014717 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1409/2804] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:23:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:23:27.019465 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930e5d5-127b-43b0-9a80-922db18226e0 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:27.036473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930e5d5-127b-43b0-9a80-922db18226e0 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:27.043297 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e040604-01cf-47e5-b133-75b8749b9aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:27.043928 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e040604-01cf-47e5-b133-75b8749b9aaf admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:27.044198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e040604-01cf-47e5-b133-75b8749b9aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:27.044448 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e040604-01cf-47e5-b133-75b8749b9aaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:27.045203 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e040604-01cf-47e5-b133-75b8749b9aaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:27.046787 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e040604-01cf-47e5-b133-75b8749b9aaf admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:27.060920 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1396/2805] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:23:27.066148 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb719afe-d2f1-4e81-acbc-a717814d213a admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:27.076910 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb719afe-d2f1-4e81-acbc-a717814d213a admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:27.081905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3785cf-1d57-4070-8f7d-1369b5afcc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:27.082577 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3785cf-1d57-4070-8f7d-1369b5afcc2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:27.082850 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3785cf-1d57-4070-8f7d-1369b5afcc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:27.083119 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3785cf-1d57-4070-8f7d-1369b5afcc2e admin admin] PATH_INFO: `/v3/projects/359e0cec64644328b0ae3c9ab0353fc5/users/ba6d13032a7e4496828609233ba829d9/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:27.100072 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3785cf-1d57-4070-8f7d-1369b5afcc2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=359e0cec64644328b0ae3c9ab0353fc5, user_id=ba6d13032a7e4496828609233ba829d9, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:27.102817 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3785cf-1d57-4070-8f7d-1369b5afcc2e admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:27.119110 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1410/2806] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:23:27 2022] PUT /identity/v3/projects/359e0cec64644328b0ae3c9ab0353fc5/users/ba6d13032a7e4496828609233ba829d9/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:27.129897 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84260ab0-e095-49c6-9aa4-5428709e0f71 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:27.130266 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84260ab0-e095-49c6-9aa4-5428709e0f71 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:27.130536 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84260ab0-e095-49c6-9aa4-5428709e0f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:27.160531 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-84260ab0-e095-49c6-9aa4-5428709e0f71 None None] MFA Rules not processed for user `ba6d13032a7e4496828609233ba829d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:23:27.161792 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-84260ab0-e095-49c6-9aa4-5428709e0f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:27.230662 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1397/2807] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:23:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:23:27.235768 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3785cf-1d57-4070-8f7d-1369b5afcc2e admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:27.307288 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3785cf-1d57-4070-8f7d-1369b5afcc2e admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:27.313118 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf4ffa-cd8f-4e97-a8c3-b010f8d7c944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:27.313768 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf4ffa-cd8f-4e97-a8c3-b010f8d7c944 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:27.314062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf4ffa-cd8f-4e97-a8c3-b010f8d7c944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:27.314328 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf4ffa-cd8f-4e97-a8c3-b010f8d7c944 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:27.314968 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cf4ffa-cd8f-4e97-a8c3-b010f8d7c944 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:27.316908 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cf4ffa-cd8f-4e97-a8c3-b010f8d7c944 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:27.348359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1411/2808] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:23:27 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:23:27.352863 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84260ab0-e095-49c6-9aa4-5428709e0f71 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:27.364624 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84260ab0-e095-49c6-9aa4-5428709e0f71 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:27.390903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:27.428717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9b6ae-b933-44cd-bc52-6841c517df17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:27.429533 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9b6ae-b933-44cd-bc52-6841c517df17 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:27.429808 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9b6ae-b933-44cd-bc52-6841c517df17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:27.430078 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9b6ae-b933-44cd-bc52-6841c517df17 admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:27.430719 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a9b6ae-b933-44cd-bc52-6841c517df17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:27.432674 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a9b6ae-b933-44cd-bc52-6841c517df17 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:27.457230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1398/2809] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:23:27 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:23:27.461776 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf4ffa-cd8f-4e97-a8c3-b010f8d7c944 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:27.473090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf4ffa-cd8f-4e97-a8c3-b010f8d7c944 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:27.478703 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b58ac-a39e-4d62-a02d-4dc3c278df30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:27.479402 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b58ac-a39e-4d62-a02d-4dc3c278df30 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:27.479676 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b58ac-a39e-4d62-a02d-4dc3c278df30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:27.479944 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b58ac-a39e-4d62-a02d-4dc3c278df30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:27.480588 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b58ac-a39e-4d62-a02d-4dc3c278df30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:27.482246 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b58ac-a39e-4d62-a02d-4dc3c278df30 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:27.493694 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1412/2810] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:23:27.498425 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9b6ae-b933-44cd-bc52-6841c517df17 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:27.509900 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9b6ae-b933-44cd-bc52-6841c517df17 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:27.515487 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc3c1c-f7e5-4e08-82b1-f92c646c8b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:27.516110 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc3c1c-f7e5-4e08-82b1-f92c646c8b0c admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:27.516385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc3c1c-f7e5-4e08-82b1-f92c646c8b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:27.516644 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc3c1c-f7e5-4e08-82b1-f92c646c8b0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:27.517531 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc3c1c-f7e5-4e08-82b1-f92c646c8b0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:27.519163 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc3c1c-f7e5-4e08-82b1-f92c646c8b0c admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:27.530919 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1399/2811] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:23:27.536046 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b58ac-a39e-4d62-a02d-4dc3c278df30 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:27.547362 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b58ac-a39e-4d62-a02d-4dc3c278df30 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:27.552861 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08665a3f-0bd8-468c-982b-c0393be0ce61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:27.553792 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08665a3f-0bd8-468c-982b-c0393be0ce61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:27.554062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08665a3f-0bd8-468c-982b-c0393be0ce61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:27.554328 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08665a3f-0bd8-468c-982b-c0393be0ce61 admin admin] PATH_INFO: `/v3/projects/68d7c38720244d5881eaa0c3df9206b3/users/0f53e2f80cca49c499d825bfb7a25d8f/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:27.572716 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08665a3f-0bd8-468c-982b-c0393be0ce61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68d7c38720244d5881eaa0c3df9206b3, user_id=0f53e2f80cca49c499d825bfb7a25d8f, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:27.575233 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08665a3f-0bd8-468c-982b-c0393be0ce61 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:27.590547 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1413/2812] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:23:27 2022] PUT /identity/v3/projects/68d7c38720244d5881eaa0c3df9206b3/users/0f53e2f80cca49c499d825bfb7a25d8f/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:27.601381 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a808c-47e1-45ac-993e-0be2f79e24c3 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:27.601665 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a808c-47e1-45ac-993e-0be2f79e24c3 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:27.601982 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a808c-47e1-45ac-993e-0be2f79e24c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:27.635991 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-eb4a808c-47e1-45ac-993e-0be2f79e24c3 None None] MFA Rules not processed for user `0f53e2f80cca49c499d825bfb7a25d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:23:27.637291 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-eb4a808c-47e1-45ac-993e-0be2f79e24c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:27.708729 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1400/2813] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:23:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:23:27.713508 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08665a3f-0bd8-468c-982b-c0393be0ce61 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:27.789895 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08665a3f-0bd8-468c-982b-c0393be0ce61 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:27.795320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b08eb-9924-489d-8853-519a10c1a2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:27.795953 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b08eb-9924-489d-8853-519a10c1a2df admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:27.796229 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b08eb-9924-489d-8853-519a10c1a2df admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:27.796499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b08eb-9924-489d-8853-519a10c1a2df admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:27.797172 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399b08eb-9924-489d-8853-519a10c1a2df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:27.799095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399b08eb-9924-489d-8853-519a10c1a2df admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:27.830082 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1414/2814] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:23:27 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:23:27.835815 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a808c-47e1-45ac-993e-0be2f79e24c3 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:27.850437 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a808c-47e1-45ac-993e-0be2f79e24c3 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:27.919210 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6e3f4-ad52-4388-88a1-a98874aaad7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:27.919838 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6e3f4-ad52-4388-88a1-a98874aaad7f admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:27.920115 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6e3f4-ad52-4388-88a1-a98874aaad7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:27.920385 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6e3f4-ad52-4388-88a1-a98874aaad7f admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:27.921050 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6e3f4-ad52-4388-88a1-a98874aaad7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:27.923062 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6e3f4-ad52-4388-88a1-a98874aaad7f admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:27.946734 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1401/2815] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:23:27 2022] POST /identity/v3/users => generated 295 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:23:27.951959 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b08eb-9924-489d-8853-519a10c1a2df admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:27.963866 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b08eb-9924-489d-8853-519a10c1a2df admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:27.971869 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59277115-973f-4c61-8a86-def48f4d4056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:27.972800 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59277115-973f-4c61-8a86-def48f4d4056 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:27.973197 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59277115-973f-4c61-8a86-def48f4d4056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:27.973711 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59277115-973f-4c61-8a86-def48f4d4056 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:27.975034 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59277115-973f-4c61-8a86-def48f4d4056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:27.977471 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59277115-973f-4c61-8a86-def48f4d4056 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:27.980688 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:27.991347 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1415/2816] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:23:27.995887 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6e3f4-ad52-4388-88a1-a98874aaad7f admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:28.011509 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6e3f4-ad52-4388-88a1-a98874aaad7f admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:28.016601 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6964556-2a2c-4299-b625-0132fe6c5053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:28.017246 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6964556-2a2c-4299-b625-0132fe6c5053 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:28.017473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6964556-2a2c-4299-b625-0132fe6c5053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:28.017740 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6964556-2a2c-4299-b625-0132fe6c5053 admin admin] PATH_INFO: `/v3/projects/0763722aa9744cc7bb9f79bdc1a3f31f/users/f574b79f4c6748da93bc2bbb57f1787a/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:28.036742 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6964556-2a2c-4299-b625-0132fe6c5053 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0763722aa9744cc7bb9f79bdc1a3f31f, user_id=f574b79f4c6748da93bc2bbb57f1787a, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:28.039600 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6964556-2a2c-4299-b625-0132fe6c5053 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:28.050189 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:28.056480 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1402/2817] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:23:27 2022] PUT /identity/v3/projects/0763722aa9744cc7bb9f79bdc1a3f31f/users/f574b79f4c6748da93bc2bbb57f1787a/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:28.061848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59277115-973f-4c61-8a86-def48f4d4056 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:28.078483 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59277115-973f-4c61-8a86-def48f4d4056 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:28.147140 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5788067-c12e-4db6-8761-b0c24761f833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:28.147731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5788067-c12e-4db6-8761-b0c24761f833 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:28.148046 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5788067-c12e-4db6-8761-b0c24761f833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:28.148419 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5788067-c12e-4db6-8761-b0c24761f833 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:28.149296 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5788067-c12e-4db6-8761-b0c24761f833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:28.151168 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5788067-c12e-4db6-8761-b0c24761f833 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:28.161938 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1416/2818] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:23:28.166705 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6964556-2a2c-4299-b625-0132fe6c5053 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:28.177647 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6964556-2a2c-4299-b625-0132fe6c5053 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:28.182731 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c94ef9-7d0d-4455-9690-57b9e4c563d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:28.183396 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c94ef9-7d0d-4455-9690-57b9e4c563d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:28.183668 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c94ef9-7d0d-4455-9690-57b9e4c563d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:28.183934 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c94ef9-7d0d-4455-9690-57b9e4c563d5 admin admin] PATH_INFO: `/v3/projects/0763722aa9744cc7bb9f79bdc1a3f31f/users/f574b79f4c6748da93bc2bbb57f1787a/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:28.186263 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c94ef9-7d0d-4455-9690-57b9e4c563d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0763722aa9744cc7bb9f79bdc1a3f31f, user_id=f574b79f4c6748da93bc2bbb57f1787a, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:28.188902 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c94ef9-7d0d-4455-9690-57b9e4c563d5 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:28.206360 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1403/2819] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:23:28 2022] PUT /identity/v3/projects/0763722aa9744cc7bb9f79bdc1a3f31f/users/f574b79f4c6748da93bc2bbb57f1787a/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:28.212436 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411540d6-472d-478f-93bf-4d08d031517f None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:28.212719 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411540d6-472d-478f-93bf-4d08d031517f None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:28.212992 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411540d6-472d-478f-93bf-4d08d031517f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:28.243354 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-411540d6-472d-478f-93bf-4d08d031517f None None] MFA Rules not processed for user `f574b79f4c6748da93bc2bbb57f1787a`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:23:28.244696 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-411540d6-472d-478f-93bf-4d08d031517f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:28.320007 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1417/2820] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:23:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:23:28.332845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c94ef9-7d0d-4455-9690-57b9e4c563d5 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:28.334956 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f4c94ef9-7d0d-4455-9690-57b9e4c563d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:28.404713 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c94ef9-7d0d-4455-9690-57b9e4c563d5 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:28.408382 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2afd32b-ac90-419a-b0a0-3d995cd81bee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:28.409014 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afd32b-ac90-419a-b0a0-3d995cd81bee service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:28.409303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afd32b-ac90-419a-b0a0-3d995cd81bee service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:28.409565 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afd32b-ac90-419a-b0a0-3d995cd81bee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:28.420643 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2afd32b-ac90-419a-b0a0-3d995cd81bee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:28.422125 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2afd32b-ac90-419a-b0a0-3d995cd81bee service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:28.432480 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1404/2821] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:23:28 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:23:28.477358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411540d6-472d-478f-93bf-4d08d031517f None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:28.478740 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-411540d6-472d-478f-93bf-4d08d031517f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:28.491029 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411540d6-472d-478f-93bf-4d08d031517f None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:28.495039 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ff5f1-9926-4f58-8623-22c78a2e9823 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:28.495640 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ff5f1-9926-4f58-8623-22c78a2e9823 service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:28.495906 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ff5f1-9926-4f58-8623-22c78a2e9823 service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:28.496169 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ff5f1-9926-4f58-8623-22c78a2e9823 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:28.558720 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ff5f1-9926-4f58-8623-22c78a2e9823 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:28.560548 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ff5f1-9926-4f58-8623-22c78a2e9823 service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:28.571023 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1418/2822] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:23:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:23:31.559248 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2afd32b-ac90-419a-b0a0-3d995cd81bee service cinder] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:31.627621 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2afd32b-ac90-419a-b0a0-3d995cd81bee service cinder] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:31.633809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33137ecc-4ac4-491d-8da6-066a0a7a0f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:31.634455 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33137ecc-4ac4-491d-8da6-066a0a7a0f10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:31.634708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33137ecc-4ac4-491d-8da6-066a0a7a0f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:31.634998 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33137ecc-4ac4-491d-8da6-066a0a7a0f10 admin admin] PATH_INFO: `/v3/users/ba6d13032a7e4496828609233ba829d9` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:31.636266 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33137ecc-4ac4-491d-8da6-066a0a7a0f10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba6d13032a7e4496828609233ba829d9)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:31.638408 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33137ecc-4ac4-491d-8da6-066a0a7a0f10 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:31.692039 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-33137ecc-4ac4-491d-8da6-066a0a7a0f10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba6d13032a7e4496828609233ba829d9'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:31.699906 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-33137ecc-4ac4-491d-8da6-066a0a7a0f10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba6d13032a7e4496828609233ba829d9'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:31.712391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-33137ecc-4ac4-491d-8da6-066a0a7a0f10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba6d13032a7e4496828609233ba829d9'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:31.713003 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-33137ecc-4ac4-491d-8da6-066a0a7a0f10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba6d13032a7e4496828609233ba829d9'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:31.720146 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-33137ecc-4ac4-491d-8da6-066a0a7a0f10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba6d13032a7e4496828609233ba829d9'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:31.726188 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1405/2823] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:23:31 2022] DELETE /identity/v3/users/ba6d13032a7e4496828609233ba829d9 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:31.732835 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa3a79-fd01-4767-9e87-f0bbe79074dc None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:31.733372 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa3a79-fd01-4767-9e87-f0bbe79074dc None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:31.733488 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa3a79-fd01-4767-9e87-f0bbe79074dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:31.754433 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-ccaa3a79-fd01-4767-9e87-f0bbe79074dc None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:23:31.755694 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-ccaa3a79-fd01-4767-9e87-f0bbe79074dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:31.769411 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:31.834192 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1419/2824] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:23:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:23:31.844746 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33137ecc-4ac4-491d-8da6-066a0a7a0f10 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:31.846198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-33137ecc-4ac4-491d-8da6-066a0a7a0f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:31.857867 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:31.907734 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33137ecc-4ac4-491d-8da6-066a0a7a0f10 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:31.911616 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93554ec1-9426-4428-bb60-62692fd78836 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:31.912223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93554ec1-9426-4428-bb60-62692fd78836 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:31.912488 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93554ec1-9426-4428-bb60-62692fd78836 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:31.912749 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93554ec1-9426-4428-bb60-62692fd78836 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:31.922756 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93554ec1-9426-4428-bb60-62692fd78836 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:31.924181 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93554ec1-9426-4428-bb60-62692fd78836 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:31.934698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1406/2825] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:23:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:23:32.265651 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa3a79-fd01-4767-9e87-f0bbe79074dc None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:32.267389 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-ccaa3a79-fd01-4767-9e87-f0bbe79074dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:32.280275 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa3a79-fd01-4767-9e87-f0bbe79074dc None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:32.285095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb70dcb-fbf5-4a87-88b7-4c9c760aa565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:32.285870 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb70dcb-fbf5-4a87-88b7-4c9c760aa565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:32.286037 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb70dcb-fbf5-4a87-88b7-4c9c760aa565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:32.286312 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb70dcb-fbf5-4a87-88b7-4c9c760aa565 admin admin] PATH_INFO: `/v3/projects/359e0cec64644328b0ae3c9ab0353fc5` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:32.287431 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb70dcb-fbf5-4a87-88b7-4c9c760aa565 admin admin] RBAC: Authorizing `identity:delete_project(project_id=359e0cec64644328b0ae3c9ab0353fc5)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:32.289381 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb70dcb-fbf5-4a87-88b7-4c9c760aa565 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:32.339130 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-efb70dcb-fbf5-4a87-88b7-4c9c760aa565 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '359e0cec64644328b0ae3c9ab0353fc5', 'request_id': 'req-efb70dcb-fbf5-4a87-88b7-4c9c760aa565'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:32.344991 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-efb70dcb-fbf5-4a87-88b7-4c9c760aa565 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '359e0cec64644328b0ae3c9ab0353fc5', 'request_id': 'req-efb70dcb-fbf5-4a87-88b7-4c9c760aa565'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:32.352378 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-efb70dcb-fbf5-4a87-88b7-4c9c760aa565 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '359e0cec64644328b0ae3c9ab0353fc5', 'request_id': 'req-efb70dcb-fbf5-4a87-88b7-4c9c760aa565'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:32.367642 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-efb70dcb-fbf5-4a87-88b7-4c9c760aa565 admin admin] The token cache is being invalidate because project 359e0cec64644328b0ae3c9ab0353fc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:23:32.367958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-efb70dcb-fbf5-4a87-88b7-4c9c760aa565 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:32.368714 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-efb70dcb-fbf5-4a87-88b7-4c9c760aa565 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:32.370376 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1420/2826] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:23:32 2022] DELETE /identity/v3/projects/359e0cec64644328b0ae3c9ab0353fc5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:32.374788 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93554ec1-9426-4428-bb60-62692fd78836 service neutron] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:32.385676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93554ec1-9426-4428-bb60-62692fd78836 service neutron] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:32.447501 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb2b01c-646e-4cdc-8b59-843efbafdd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:32.448166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2b01c-646e-4cdc-8b59-843efbafdd8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:32.448437 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2b01c-646e-4cdc-8b59-843efbafdd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:32.448699 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2b01c-646e-4cdc-8b59-843efbafdd8a admin admin] PATH_INFO: `/v3/users/0f53e2f80cca49c499d825bfb7a25d8f` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:32.449871 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb2b01c-646e-4cdc-8b59-843efbafdd8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f53e2f80cca49c499d825bfb7a25d8f)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:32.451872 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb2b01c-646e-4cdc-8b59-843efbafdd8a admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:32.502054 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cdb2b01c-646e-4cdc-8b59-843efbafdd8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f53e2f80cca49c499d825bfb7a25d8f'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:32.511041 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cdb2b01c-646e-4cdc-8b59-843efbafdd8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f53e2f80cca49c499d825bfb7a25d8f'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:32.523778 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cdb2b01c-646e-4cdc-8b59-843efbafdd8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f53e2f80cca49c499d825bfb7a25d8f'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:32.525275 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cdb2b01c-646e-4cdc-8b59-843efbafdd8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f53e2f80cca49c499d825bfb7a25d8f'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:32.533710 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-cdb2b01c-646e-4cdc-8b59-843efbafdd8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f53e2f80cca49c499d825bfb7a25d8f'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:32.539908 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1407/2827] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:23:32 2022] DELETE /identity/v3/users/0f53e2f80cca49c499d825bfb7a25d8f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:23:32.839548 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb70dcb-fbf5-4a87-88b7-4c9c760aa565 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:32.841373 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-efb70dcb-fbf5-4a87-88b7-4c9c760aa565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:32.863054 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb70dcb-fbf5-4a87-88b7-4c9c760aa565 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:32.935301 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a0db47-d6dc-40e2-af29-08b7e5f11880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:32.936140 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0db47-d6dc-40e2-af29-08b7e5f11880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:32.936405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0db47-d6dc-40e2-af29-08b7e5f11880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:32.936686 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0db47-d6dc-40e2-af29-08b7e5f11880 admin admin] PATH_INFO: `/v3/projects/68d7c38720244d5881eaa0c3df9206b3` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:32.937958 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a0db47-d6dc-40e2-af29-08b7e5f11880 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68d7c38720244d5881eaa0c3df9206b3)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:32.940038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a0db47-d6dc-40e2-af29-08b7e5f11880 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:32.988633 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:32.993764 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-d4a0db47-d6dc-40e2-af29-08b7e5f11880 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68d7c38720244d5881eaa0c3df9206b3', 'request_id': 'req-d4a0db47-d6dc-40e2-af29-08b7e5f11880'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:33.002158 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-d4a0db47-d6dc-40e2-af29-08b7e5f11880 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68d7c38720244d5881eaa0c3df9206b3', 'request_id': 'req-d4a0db47-d6dc-40e2-af29-08b7e5f11880'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:33.011544 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-d4a0db47-d6dc-40e2-af29-08b7e5f11880 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68d7c38720244d5881eaa0c3df9206b3', 'request_id': 'req-d4a0db47-d6dc-40e2-af29-08b7e5f11880'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:33.027763 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-d4a0db47-d6dc-40e2-af29-08b7e5f11880 admin admin] The token cache is being invalidate because project 68d7c38720244d5881eaa0c3df9206b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:23:33.028060 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-d4a0db47-d6dc-40e2-af29-08b7e5f11880 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:33.028764 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-d4a0db47-d6dc-40e2-af29-08b7e5f11880 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:33.030513 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1421/2828] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:23:32 2022] DELETE /identity/v3/projects/68d7c38720244d5881eaa0c3df9206b3 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:33.034806 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb2b01c-646e-4cdc-8b59-843efbafdd8a admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:33.036447 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-cdb2b01c-646e-4cdc-8b59-843efbafdd8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:33.050258 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb2b01c-646e-4cdc-8b59-843efbafdd8a admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:33.081946 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:33.116294 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd8a9d-a8dc-46b3-9aaf-c9b35ce39a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:33.116938 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd8a9d-a8dc-46b3-9aaf-c9b35ce39a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:33.117263 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd8a9d-a8dc-46b3-9aaf-c9b35ce39a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:33.117535 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd8a9d-a8dc-46b3-9aaf-c9b35ce39a17 admin admin] PATH_INFO: `/v3/users/f574b79f4c6748da93bc2bbb57f1787a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:33.118844 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dd8a9d-a8dc-46b3-9aaf-c9b35ce39a17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f574b79f4c6748da93bc2bbb57f1787a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:33.120784 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dd8a9d-a8dc-46b3-9aaf-c9b35ce39a17 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:33.175847 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-24dd8a9d-a8dc-46b3-9aaf-c9b35ce39a17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f574b79f4c6748da93bc2bbb57f1787a'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:33.185349 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-24dd8a9d-a8dc-46b3-9aaf-c9b35ce39a17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f574b79f4c6748da93bc2bbb57f1787a'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:33.197948 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-24dd8a9d-a8dc-46b3-9aaf-c9b35ce39a17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f574b79f4c6748da93bc2bbb57f1787a'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:33.198666 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-24dd8a9d-a8dc-46b3-9aaf-c9b35ce39a17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f574b79f4c6748da93bc2bbb57f1787a'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:33.206430 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-24dd8a9d-a8dc-46b3-9aaf-c9b35ce39a17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f574b79f4c6748da93bc2bbb57f1787a'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:33.213516 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1408/2829] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:23:33 2022] DELETE /identity/v3/users/f574b79f4c6748da93bc2bbb57f1787a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:33.514095 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a0db47-d6dc-40e2-af29-08b7e5f11880 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:33.515710 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-d4a0db47-d6dc-40e2-af29-08b7e5f11880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:33.531147 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a0db47-d6dc-40e2-af29-08b7e5f11880 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:33.602630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb261069-75fe-4e6a-a8db-21df04b66aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:33.602630 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb261069-75fe-4e6a-a8db-21df04b66aba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:33.603116 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb261069-75fe-4e6a-a8db-21df04b66aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:33.603255 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb261069-75fe-4e6a-a8db-21df04b66aba admin admin] PATH_INFO: `/v3/projects/0763722aa9744cc7bb9f79bdc1a3f31f` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:33.604359 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb261069-75fe-4e6a-a8db-21df04b66aba admin admin] RBAC: Authorizing `identity:delete_project(project_id=0763722aa9744cc7bb9f79bdc1a3f31f)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:33.606361 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb261069-75fe-4e6a-a8db-21df04b66aba admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:33.656643 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-cb261069-75fe-4e6a-a8db-21df04b66aba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0763722aa9744cc7bb9f79bdc1a3f31f', 'request_id': 'req-cb261069-75fe-4e6a-a8db-21df04b66aba'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:33.663324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-cb261069-75fe-4e6a-a8db-21df04b66aba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0763722aa9744cc7bb9f79bdc1a3f31f', 'request_id': 'req-cb261069-75fe-4e6a-a8db-21df04b66aba'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:33.670120 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-cb261069-75fe-4e6a-a8db-21df04b66aba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0763722aa9744cc7bb9f79bdc1a3f31f', 'request_id': 'req-cb261069-75fe-4e6a-a8db-21df04b66aba'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:33.685216 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-cb261069-75fe-4e6a-a8db-21df04b66aba admin admin] The token cache is being invalidate because project 0763722aa9744cc7bb9f79bdc1a3f31f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:23:33.685507 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-cb261069-75fe-4e6a-a8db-21df04b66aba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:33.686211 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-cb261069-75fe-4e6a-a8db-21df04b66aba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:33.687781 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1422/2830] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:23:33 2022] DELETE /identity/v3/projects/0763722aa9744cc7bb9f79bdc1a3f31f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:33.693920 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e92b65-a520-4bcb-9ed4-57b40aa78c2d None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:33.694208 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e92b65-a520-4bcb-9ed4-57b40aa78c2d None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:33.694478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e92b65-a520-4bcb-9ed4-57b40aa78c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:33.716727 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-44e92b65-a520-4bcb-9ed4-57b40aa78c2d None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:23:33.717977 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-44e92b65-a520-4bcb-9ed4-57b40aa78c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:33.785204 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1409/2831] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:23:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:23:33.793839 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1cae8c-7ce4-4a55-b5cf-553dfb68fc3b None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:33.794162 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1cae8c-7ce4-4a55-b5cf-553dfb68fc3b None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:33.794514 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1cae8c-7ce4-4a55-b5cf-553dfb68fc3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:33.818382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-de1cae8c-7ce4-4a55-b5cf-553dfb68fc3b None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:23:33.820516 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-de1cae8c-7ce4-4a55-b5cf-553dfb68fc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:33.834385 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1423/2832] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:23:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:23:33.839054 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e92b65-a520-4bcb-9ed4-57b40aa78c2d None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:33.849751 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e92b65-a520-4bcb-9ed4-57b40aa78c2d None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:33.855744 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3822ec8-ca78-4e0b-aa3a-7511dac8b8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:33.856694 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3822ec8-ca78-4e0b-aa3a-7511dac8b8e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:33.857074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3822ec8-ca78-4e0b-aa3a-7511dac8b8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:33.857476 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3822ec8-ca78-4e0b-aa3a-7511dac8b8e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:33.858239 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3822ec8-ca78-4e0b-aa3a-7511dac8b8e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:23:33.858646 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3822ec8-ca78-4e0b-aa3a-7511dac8b8e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:33.860505 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3822ec8-ca78-4e0b-aa3a-7511dac8b8e3 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:33.879965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1410/2833] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:23:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:23:33.886036 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1cae8c-7ce4-4a55-b5cf-553dfb68fc3b None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:33.896760 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1cae8c-7ce4-4a55-b5cf-553dfb68fc3b None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:33.901718 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97f118-4197-4210-9d22-a1707277ad53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:33.902341 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97f118-4197-4210-9d22-a1707277ad53 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:33.902615 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97f118-4197-4210-9d22-a1707277ad53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:33.902882 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97f118-4197-4210-9d22-a1707277ad53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:33.903571 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b97f118-4197-4210-9d22-a1707277ad53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:33.905573 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b97f118-4197-4210-9d22-a1707277ad53 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:33.943490 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1424/2834] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:23:33 2022] POST /identity/v3/projects => generated 367 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:23:33.948437 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3822ec8-ca78-4e0b-aa3a-7511dac8b8e3 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:33.959274 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3822ec8-ca78-4e0b-aa3a-7511dac8b8e3 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:33.993155 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:34.020176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25829c-22bf-406d-b5dc-1fee01e64f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:34.020843 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25829c-22bf-406d-b5dc-1fee01e64f80 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:34.021163 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25829c-22bf-406d-b5dc-1fee01e64f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:34.021548 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25829c-22bf-406d-b5dc-1fee01e64f80 admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:34.022225 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed25829c-22bf-406d-b5dc-1fee01e64f80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:34.024354 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed25829c-22bf-406d-b5dc-1fee01e64f80 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:34.048664 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1411/2835] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:23:33 2022] POST /identity/v3/users => generated 284 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:23:34.053583 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97f118-4197-4210-9d22-a1707277ad53 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:34.067392 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97f118-4197-4210-9d22-a1707277ad53 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:34.072408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf86a2f-e7b3-4ec7-a015-078c96b1d068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:34.073038 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf86a2f-e7b3-4ec7-a015-078c96b1d068 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:34.073344 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf86a2f-e7b3-4ec7-a015-078c96b1d068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:34.073613 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf86a2f-e7b3-4ec7-a015-078c96b1d068 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:34.074284 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf86a2f-e7b3-4ec7-a015-078c96b1d068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:34.076030 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf86a2f-e7b3-4ec7-a015-078c96b1d068 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:34.087276 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1425/2836] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:23:34.092458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25829c-22bf-406d-b5dc-1fee01e64f80 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:34.103159 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25829c-22bf-406d-b5dc-1fee01e64f80 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:34.108054 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6d7d6-7176-4443-a085-6e150e12793b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:34.108673 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6d7d6-7176-4443-a085-6e150e12793b admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:34.108944 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6d7d6-7176-4443-a085-6e150e12793b admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:34.109234 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6d7d6-7176-4443-a085-6e150e12793b admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:34.109887 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c6d7d6-7176-4443-a085-6e150e12793b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:34.111520 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c6d7d6-7176-4443-a085-6e150e12793b admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:34.123113 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1412/2837] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:23:34.127868 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf86a2f-e7b3-4ec7-a015-078c96b1d068 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:34.139033 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf86a2f-e7b3-4ec7-a015-078c96b1d068 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:34.144020 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd187b52-14ca-4e94-bfa7-b57ec739cc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:34.144713 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd187b52-14ca-4e94-bfa7-b57ec739cc67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:34.144987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd187b52-14ca-4e94-bfa7-b57ec739cc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:34.145306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd187b52-14ca-4e94-bfa7-b57ec739cc67 admin admin] PATH_INFO: `/v3/projects/8b6d4834f64c4ccab4445f6b543fae0b/users/085b442312aa43b48030095bc3a81c2b/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:34.164594 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd187b52-14ca-4e94-bfa7-b57ec739cc67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b6d4834f64c4ccab4445f6b543fae0b, user_id=085b442312aa43b48030095bc3a81c2b, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:34.169460 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd187b52-14ca-4e94-bfa7-b57ec739cc67 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:34.181280 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:34.188711 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1426/2838] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:23:34 2022] PUT /identity/v3/projects/8b6d4834f64c4ccab4445f6b543fae0b/users/085b442312aa43b48030095bc3a81c2b/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:34.203805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed5d6a-0ae2-4f35-87b0-f7cc3f6c9579 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:34.204083 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed5d6a-0ae2-4f35-87b0-f7cc3f6c9579 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:34.204348 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed5d6a-0ae2-4f35-87b0-f7cc3f6c9579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:34.238096 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-02ed5d6a-0ae2-4f35-87b0-f7cc3f6c9579 None None] MFA Rules not processed for user `085b442312aa43b48030095bc3a81c2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:23:34.239315 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-02ed5d6a-0ae2-4f35-87b0-f7cc3f6c9579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:34.314272 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1413/2839] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:23:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:23:34.318847 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd187b52-14ca-4e94-bfa7-b57ec739cc67 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:34.387255 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd187b52-14ca-4e94-bfa7-b57ec739cc67 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:34.392701 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8843491-957f-4e01-b260-3315675c6265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:34.393357 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8843491-957f-4e01-b260-3315675c6265 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:34.393631 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8843491-957f-4e01-b260-3315675c6265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:34.393899 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8843491-957f-4e01-b260-3315675c6265 admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:34.394528 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8843491-957f-4e01-b260-3315675c6265 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:34.396437 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8843491-957f-4e01-b260-3315675c6265 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:34.427601 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1427/2840] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:23:34 2022] POST /identity/v3/projects => generated 367 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:23:34.432202 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed5d6a-0ae2-4f35-87b0-f7cc3f6c9579 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:34.443632 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed5d6a-0ae2-4f35-87b0-f7cc3f6c9579 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:34.504030 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82388f89-f60e-4533-aed5-17bd1278080f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:34.504798 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82388f89-f60e-4533-aed5-17bd1278080f admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:34.505300 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82388f89-f60e-4533-aed5-17bd1278080f admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:34.505604 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82388f89-f60e-4533-aed5-17bd1278080f admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:34.506272 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82388f89-f60e-4533-aed5-17bd1278080f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:34.508311 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82388f89-f60e-4533-aed5-17bd1278080f admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:34.539458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1414/2841] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:23:34 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:23:34.545533 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8843491-957f-4e01-b260-3315675c6265 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:34.556884 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8843491-957f-4e01-b260-3315675c6265 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:34.562931 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6c26b-cb23-40b1-8471-4d76647e869a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:34.563564 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6c26b-cb23-40b1-8471-4d76647e869a admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:34.563834 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6c26b-cb23-40b1-8471-4d76647e869a admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:34.564093 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6c26b-cb23-40b1-8471-4d76647e869a admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:34.564819 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e6c26b-cb23-40b1-8471-4d76647e869a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:34.566490 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e6c26b-cb23-40b1-8471-4d76647e869a admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:34.578655 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1428/2842] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:23:34.583496 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82388f89-f60e-4533-aed5-17bd1278080f admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:34.600119 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82388f89-f60e-4533-aed5-17bd1278080f admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:34.608117 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90169a78-9442-45c3-a02e-962e15de02bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:34.609261 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90169a78-9442-45c3-a02e-962e15de02bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:34.609727 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90169a78-9442-45c3-a02e-962e15de02bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:34.610222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90169a78-9442-45c3-a02e-962e15de02bd admin admin] PATH_INFO: `/v3/projects/68fde060a3b345439c001efd10566267/users/bdd6a11d037e4c429dece62fe5756765/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:34.630816 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90169a78-9442-45c3-a02e-962e15de02bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=68fde060a3b345439c001efd10566267, user_id=bdd6a11d037e4c429dece62fe5756765, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:34.633549 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90169a78-9442-45c3-a02e-962e15de02bd admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:34.649305 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1415/2843] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:23:34 2022] PUT /identity/v3/projects/68fde060a3b345439c001efd10566267/users/bdd6a11d037e4c429dece62fe5756765/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:34.653772 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6c26b-cb23-40b1-8471-4d76647e869a admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:34.664932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6c26b-cb23-40b1-8471-4d76647e869a admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:34.727242 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab84ee4-1386-4078-bb44-72bbdc5aebab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:34.727859 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab84ee4-1386-4078-bb44-72bbdc5aebab admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:34.728133 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab84ee4-1386-4078-bb44-72bbdc5aebab admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:34.728402 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab84ee4-1386-4078-bb44-72bbdc5aebab admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:34.729039 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab84ee4-1386-4078-bb44-72bbdc5aebab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:34.730665 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab84ee4-1386-4078-bb44-72bbdc5aebab admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:34.741447 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1429/2844] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:23:34.745941 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90169a78-9442-45c3-a02e-962e15de02bd admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:34.756996 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90169a78-9442-45c3-a02e-962e15de02bd admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:34.762164 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8859d-9603-448f-a401-aadab8c6868d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:34.762899 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8859d-9603-448f-a401-aadab8c6868d admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:34.763169 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8859d-9603-448f-a401-aadab8c6868d admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:34.763417 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8859d-9603-448f-a401-aadab8c6868d admin admin] PATH_INFO: `/v3/projects/68fde060a3b345439c001efd10566267/users/bdd6a11d037e4c429dece62fe5756765/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:34.765403 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d8859d-9603-448f-a401-aadab8c6868d admin admin] RBAC: Authorizing `identity:create_grant(project_id=68fde060a3b345439c001efd10566267, user_id=bdd6a11d037e4c429dece62fe5756765, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:34.768074 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d8859d-9603-448f-a401-aadab8c6868d admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:34.774461 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:34.782326 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1416/2845] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:23:34 2022] PUT /identity/v3/projects/68fde060a3b345439c001efd10566267/users/bdd6a11d037e4c429dece62fe5756765/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:34.793634 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee32c31-3b36-47e0-9d46-50a5704e39f6 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:34.794075 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee32c31-3b36-47e0-9d46-50a5704e39f6 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:34.794345 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee32c31-3b36-47e0-9d46-50a5704e39f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:34.824765 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-6ee32c31-3b36-47e0-9d46-50a5704e39f6 None None] MFA Rules not processed for user `bdd6a11d037e4c429dece62fe5756765`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:23:34.826046 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-6ee32c31-3b36-47e0-9d46-50a5704e39f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:34.898200 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:34.910814 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1430/2846] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:23:34 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:23:34.923052 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8859d-9603-448f-a401-aadab8c6868d admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:34.924825 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-42d8859d-9603-448f-a401-aadab8c6868d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:34.991951 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8859d-9603-448f-a401-aadab8c6868d admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:34.995707 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c26da4-3f09-4f2c-ae7c-cdcbfe34c7e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:34.996345 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c26da4-3f09-4f2c-ae7c-cdcbfe34c7e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:34.996617 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c26da4-3f09-4f2c-ae7c-cdcbfe34c7e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:34.996887 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c26da4-3f09-4f2c-ae7c-cdcbfe34c7e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:35.007755 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c26da4-3f09-4f2c-ae7c-cdcbfe34c7e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:35.009268 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c26da4-3f09-4f2c-ae7c-cdcbfe34c7e5 service cinder] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:35.019689 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1417/2847] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:23:34 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:23:35.120790 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee32c31-3b36-47e0-9d46-50a5704e39f6 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:35.132022 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee32c31-3b36-47e0-9d46-50a5704e39f6 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:35.136391 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c506953-be3b-45e4-89d1-743af8fae4e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b97e2542db54900a99f6f44b78a5e24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:35.137000 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c506953-be3b-45e4-89d1-743af8fae4e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:35.137306 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c506953-be3b-45e4-89d1-743af8fae4e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:35.137572 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c506953-be3b-45e4-89d1-743af8fae4e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:35.201232 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c506953-be3b-45e4-89d1-743af8fae4e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:35.202691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c506953-be3b-45e4-89d1-743af8fae4e7 service cinder] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:35.212836 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1431/2848] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:23:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:23:38.196580 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c26da4-3f09-4f2c-ae7c-cdcbfe34c7e5 service cinder] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:38.198108 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c4c26da4-3f09-4f2c-ae7c-cdcbfe34c7e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:38.299943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c26da4-3f09-4f2c-ae7c-cdcbfe34c7e5 service cinder] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:38.303640 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf175ecb-0a7a-4616-82ca-9cc4b333e71c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:38.304473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf175ecb-0a7a-4616-82ca-9cc4b333e71c service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:38.304740 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf175ecb-0a7a-4616-82ca-9cc4b333e71c service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:38.305122 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf175ecb-0a7a-4616-82ca-9cc4b333e71c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:38.306929 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-cf175ecb-0a7a-4616-82ca-9cc4b333e71c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:38.318948 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf175ecb-0a7a-4616-82ca-9cc4b333e71c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:38.319903 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf175ecb-0a7a-4616-82ca-9cc4b333e71c service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:38.331190 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1418/2849] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:23:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:23:40.521996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c506953-be3b-45e4-89d1-743af8fae4e7 service cinder] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:40.605231 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c506953-be3b-45e4-89d1-743af8fae4e7 service cinder] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:40.611044 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f80ed-bf7c-40ad-a0c6-4d5a272aab2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:40.612139 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f80ed-bf7c-40ad-a0c6-4d5a272aab2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:40.612632 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f80ed-bf7c-40ad-a0c6-4d5a272aab2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:40.613014 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f80ed-bf7c-40ad-a0c6-4d5a272aab2b admin admin] PATH_INFO: `/v3/users/085b442312aa43b48030095bc3a81c2b` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:40.615101 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f80ed-bf7c-40ad-a0c6-4d5a272aab2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=085b442312aa43b48030095bc3a81c2b)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:40.618902 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f80ed-bf7c-40ad-a0c6-4d5a272aab2b admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:40.683672 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5e6f80ed-bf7c-40ad-a0c6-4d5a272aab2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '085b442312aa43b48030095bc3a81c2b'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:40.693178 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5e6f80ed-bf7c-40ad-a0c6-4d5a272aab2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '085b442312aa43b48030095bc3a81c2b'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:40.711482 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5e6f80ed-bf7c-40ad-a0c6-4d5a272aab2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '085b442312aa43b48030095bc3a81c2b'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:40.712249 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5e6f80ed-bf7c-40ad-a0c6-4d5a272aab2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '085b442312aa43b48030095bc3a81c2b'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:40.721161 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5e6f80ed-bf7c-40ad-a0c6-4d5a272aab2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '085b442312aa43b48030095bc3a81c2b'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:40.727791 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1432/2850] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:23:40 2022] DELETE /identity/v3/users/085b442312aa43b48030095bc3a81c2b => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:40.732809 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d26e70-68ee-4d18-9018-b9b3c54e799b None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:40.733414 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d26e70-68ee-4d18-9018-b9b3c54e799b None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:40.733414 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d26e70-68ee-4d18-9018-b9b3c54e799b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:40.757060 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-e8d26e70-68ee-4d18-9018-b9b3c54e799b None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:23:40.758343 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-e8d26e70-68ee-4d18-9018-b9b3c54e799b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:40.831900 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1419/2851] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:23:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:23:40.843164 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f80ed-bf7c-40ad-a0c6-4d5a272aab2b admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:40.845059 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-5e6f80ed-bf7c-40ad-a0c6-4d5a272aab2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:40.860067 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:40.914401 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f80ed-bf7c-40ad-a0c6-4d5a272aab2b admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:40.919645 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fadd8f-7032-4df7-9b54-fbcd21bc1976 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:40.920287 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fadd8f-7032-4df7-9b54-fbcd21bc1976 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:40.920548 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fadd8f-7032-4df7-9b54-fbcd21bc1976 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:40.920823 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fadd8f-7032-4df7-9b54-fbcd21bc1976 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:40.932859 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fadd8f-7032-4df7-9b54-fbcd21bc1976 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:40.934361 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fadd8f-7032-4df7-9b54-fbcd21bc1976 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:40.945805 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1433/2852] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:23:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:23:41.339811 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d26e70-68ee-4d18-9018-b9b3c54e799b None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:41.341458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-e8d26e70-68ee-4d18-9018-b9b3c54e799b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:41.358783 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d26e70-68ee-4d18-9018-b9b3c54e799b None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:41.366776 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d5b1e-c75a-4316-ac81-0dac0689f7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:41.367850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d5b1e-c75a-4316-ac81-0dac0689f7e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:41.368437 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d5b1e-c75a-4316-ac81-0dac0689f7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:41.368773 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d5b1e-c75a-4316-ac81-0dac0689f7e8 admin admin] PATH_INFO: `/v3/projects/8b6d4834f64c4ccab4445f6b543fae0b` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:41.370800 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d5b1e-c75a-4316-ac81-0dac0689f7e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b6d4834f64c4ccab4445f6b543fae0b)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:41.374955 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d5b1e-c75a-4316-ac81-0dac0689f7e8 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:41.424801 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:41.436740 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-ac7d5b1e-c75a-4316-ac81-0dac0689f7e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b6d4834f64c4ccab4445f6b543fae0b', 'request_id': 'req-ac7d5b1e-c75a-4316-ac81-0dac0689f7e8'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:41.446803 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-ac7d5b1e-c75a-4316-ac81-0dac0689f7e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b6d4834f64c4ccab4445f6b543fae0b', 'request_id': 'req-ac7d5b1e-c75a-4316-ac81-0dac0689f7e8'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:41.453789 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-ac7d5b1e-c75a-4316-ac81-0dac0689f7e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b6d4834f64c4ccab4445f6b543fae0b', 'request_id': 'req-ac7d5b1e-c75a-4316-ac81-0dac0689f7e8'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:41.466494 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-ac7d5b1e-c75a-4316-ac81-0dac0689f7e8 admin admin] The token cache is being invalidate because project 8b6d4834f64c4ccab4445f6b543fae0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:23:41.466845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-ac7d5b1e-c75a-4316-ac81-0dac0689f7e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:41.467472 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-ac7d5b1e-c75a-4316-ac81-0dac0689f7e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:41.469027 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1420/2853] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:23:41 2022] DELETE /identity/v3/projects/8b6d4834f64c4ccab4445f6b543fae0b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:41.473808 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fadd8f-7032-4df7-9b54-fbcd21bc1976 service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:41.487132 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fadd8f-7032-4df7-9b54-fbcd21bc1976 service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:41.573043 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f0403-6630-470d-9c31-767f46a23036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:41.573985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f0403-6630-470d-9c31-767f46a23036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:41.574160 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f0403-6630-470d-9c31-767f46a23036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:41.574361 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f0403-6630-470d-9c31-767f46a23036 admin admin] PATH_INFO: `/v3/users/bdd6a11d037e4c429dece62fe5756765` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:41.575865 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385f0403-6630-470d-9c31-767f46a23036 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdd6a11d037e4c429dece62fe5756765)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:41.577967 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385f0403-6630-470d-9c31-767f46a23036 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:41.632635 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-385f0403-6630-470d-9c31-767f46a23036 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdd6a11d037e4c429dece62fe5756765'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:41.641438 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-385f0403-6630-470d-9c31-767f46a23036 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdd6a11d037e4c429dece62fe5756765'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:41.656846 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-385f0403-6630-470d-9c31-767f46a23036 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdd6a11d037e4c429dece62fe5756765'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:41.657620 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-385f0403-6630-470d-9c31-767f46a23036 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdd6a11d037e4c429dece62fe5756765'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:41.665447 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-385f0403-6630-470d-9c31-767f46a23036 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdd6a11d037e4c429dece62fe5756765'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:41.671844 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1434/2854] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:23:41 2022] DELETE /identity/v3/users/bdd6a11d037e4c429dece62fe5756765 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:41.976630 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d5b1e-c75a-4316-ac81-0dac0689f7e8 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:41.978478 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-ac7d5b1e-c75a-4316-ac81-0dac0689f7e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:41.996611 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d5b1e-c75a-4316-ac81-0dac0689f7e8 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:42.074753 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e42a22-0031-4de9-a641-239c0acaa67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:42.075404 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e42a22-0031-4de9-a641-239c0acaa67d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:42.075675 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e42a22-0031-4de9-a641-239c0acaa67d admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:42.075938 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e42a22-0031-4de9-a641-239c0acaa67d admin admin] PATH_INFO: `/v3/projects/68fde060a3b345439c001efd10566267` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:42.077118 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e42a22-0031-4de9-a641-239c0acaa67d admin admin] RBAC: Authorizing `identity:delete_project(project_id=68fde060a3b345439c001efd10566267)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:42.079122 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e42a22-0031-4de9-a641-239c0acaa67d admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:42.130449 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-e8e42a22-0031-4de9-a641-239c0acaa67d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68fde060a3b345439c001efd10566267', 'request_id': 'req-e8e42a22-0031-4de9-a641-239c0acaa67d'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:42.141100 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-e8e42a22-0031-4de9-a641-239c0acaa67d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68fde060a3b345439c001efd10566267', 'request_id': 'req-e8e42a22-0031-4de9-a641-239c0acaa67d'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:42.150229 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-e8e42a22-0031-4de9-a641-239c0acaa67d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68fde060a3b345439c001efd10566267', 'request_id': 'req-e8e42a22-0031-4de9-a641-239c0acaa67d'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:42.166609 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-e8e42a22-0031-4de9-a641-239c0acaa67d admin admin] The token cache is being invalidate because project 68fde060a3b345439c001efd10566267 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81684) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:23:42.167120 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-e8e42a22-0031-4de9-a641-239c0acaa67d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:42.168295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-e8e42a22-0031-4de9-a641-239c0acaa67d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:23:42.170345 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1421/2855] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:23:41 2022] DELETE /identity/v3/projects/68fde060a3b345439c001efd10566267 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:42.175736 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0a568-edfc-4b03-8c5e-672946f33d20 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:42.176016 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0a568-edfc-4b03-8c5e-672946f33d20 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:42.176316 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0a568-edfc-4b03-8c5e-672946f33d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:42.197522 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-86e0a568-edfc-4b03-8c5e-672946f33d20 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:23:42.198848 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-86e0a568-edfc-4b03-8c5e-672946f33d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:42.237255 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:42.272796 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1435/2856] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:23:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:23:42.281426 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524992a2-96e8-4c3a-bf5d-14c49130a5e4 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:42.281735 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524992a2-96e8-4c3a-bf5d-14c49130a5e4 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:42.282005 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524992a2-96e8-4c3a-bf5d-14c49130a5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:42.304768 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-524992a2-96e8-4c3a-bf5d-14c49130a5e4 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:23:42.306110 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-524992a2-96e8-4c3a-bf5d-14c49130a5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:42.314715 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1422/2857] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:23:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:23:42.319176 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e0a568-edfc-4b03-8c5e-672946f33d20 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:42.329003 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e0a568-edfc-4b03-8c5e-672946f33d20 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:42.333981 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc3d40-8522-46a4-93dd-76e4f3d8b76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:42.334587 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc3d40-8522-46a4-93dd-76e4f3d8b76b admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:42.335012 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc3d40-8522-46a4-93dd-76e4f3d8b76b admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:42.335184 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc3d40-8522-46a4-93dd-76e4f3d8b76b admin admin] PATH_INFO: `/v3/domains` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:42.335759 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfc3d40-8522-46a4-93dd-76e4f3d8b76b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81683) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:23:42.336063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfc3d40-8522-46a4-93dd-76e4f3d8b76b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:42.337786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfc3d40-8522-46a4-93dd-76e4f3d8b76b admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:42.351175 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1436/2858] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:23:42.355857 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524992a2-96e8-4c3a-bf5d-14c49130a5e4 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:42.368391 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524992a2-96e8-4c3a-bf5d-14c49130a5e4 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:42.374370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e236ede1-c45a-4838-a98f-42a292928fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:42.375045 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236ede1-c45a-4838-a98f-42a292928fc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:42.375355 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236ede1-c45a-4838-a98f-42a292928fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:42.375743 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236ede1-c45a-4838-a98f-42a292928fc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:42.376436 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e236ede1-c45a-4838-a98f-42a292928fc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:42.378678 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e236ede1-c45a-4838-a98f-42a292928fc4 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:42.411238 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1423/2859] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:23:42 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:23:42.415902 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc3d40-8522-46a4-93dd-76e4f3d8b76b admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:42.427499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc3d40-8522-46a4-93dd-76e4f3d8b76b admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:42.492847 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09825bb5-2da4-4499-8439-66ac51a37503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:42.493380 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09825bb5-2da4-4499-8439-66ac51a37503 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:42.493655 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09825bb5-2da4-4499-8439-66ac51a37503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:42.493930 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09825bb5-2da4-4499-8439-66ac51a37503 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:42.494605 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09825bb5-2da4-4499-8439-66ac51a37503 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:42.496621 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09825bb5-2da4-4499-8439-66ac51a37503 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:42.522541 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1437/2860] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:23:42 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:23:42.527349 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e236ede1-c45a-4838-a98f-42a292928fc4 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:42.539253 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e236ede1-c45a-4838-a98f-42a292928fc4 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:42.544719 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fba8f4-3db1-4073-a17c-38efbe64865c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:42.545401 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fba8f4-3db1-4073-a17c-38efbe64865c admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:42.545676 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fba8f4-3db1-4073-a17c-38efbe64865c admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:42.545941 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fba8f4-3db1-4073-a17c-38efbe64865c admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:42.546605 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fba8f4-3db1-4073-a17c-38efbe64865c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:42.548273 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fba8f4-3db1-4073-a17c-38efbe64865c admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:42.560223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1424/2861] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:23:42.564954 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09825bb5-2da4-4499-8439-66ac51a37503 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:42.576619 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09825bb5-2da4-4499-8439-66ac51a37503 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:42.582647 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae5547-f452-40cd-a931-3e3f34ab3706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:42.583463 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae5547-f452-40cd-a931-3e3f34ab3706 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:42.583795 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae5547-f452-40cd-a931-3e3f34ab3706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:42.584100 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae5547-f452-40cd-a931-3e3f34ab3706 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:42.584816 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae5547-f452-40cd-a931-3e3f34ab3706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:42.586723 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae5547-f452-40cd-a931-3e3f34ab3706 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:42.599249 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1438/2862] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:23:42.603797 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fba8f4-3db1-4073-a17c-38efbe64865c admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:42.615075 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fba8f4-3db1-4073-a17c-38efbe64865c admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:42.620427 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f7a66-0b03-431b-9119-179a981ffc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:42.621105 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f7a66-0b03-431b-9119-179a981ffc7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:42.621411 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f7a66-0b03-431b-9119-179a981ffc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:42.621677 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f7a66-0b03-431b-9119-179a981ffc7b admin admin] PATH_INFO: `/v3/projects/5b8c0728fc2e4e54ab90bbce9be73862/users/53fd875fa50e4bee8bb7cece0efe4e9b/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:42.639414 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f7a66-0b03-431b-9119-179a981ffc7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b8c0728fc2e4e54ab90bbce9be73862, user_id=53fd875fa50e4bee8bb7cece0efe4e9b, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:42.642166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f7a66-0b03-431b-9119-179a981ffc7b admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:42.644620 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:42.660668 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1425/2863] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:23:42 2022] PUT /identity/v3/projects/5b8c0728fc2e4e54ab90bbce9be73862/users/53fd875fa50e4bee8bb7cece0efe4e9b/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:42.666515 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cedd342-8922-4de3-8775-bb75389f949f None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:42.666785 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cedd342-8922-4de3-8775-bb75389f949f None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:42.667056 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cedd342-8922-4de3-8775-bb75389f949f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:42.687673 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-3cedd342-8922-4de3-8775-bb75389f949f None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:23:42.688885 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-3cedd342-8922-4de3-8775-bb75389f949f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:42.755314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1439/2864] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:23:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:23:42.765666 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f7a66-0b03-431b-9119-179a981ffc7b admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:42.767470 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-8b2f7a66-0b03-431b-9119-179a981ffc7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:42.828354 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f7a66-0b03-431b-9119-179a981ffc7b admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:42.832546 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db47f21-16af-4928-8816-7cf9b567e470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:42.833199 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db47f21-16af-4928-8816-7cf9b567e470 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:42.833495 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db47f21-16af-4928-8816-7cf9b567e470 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:42.833756 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db47f21-16af-4928-8816-7cf9b567e470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:42.844640 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db47f21-16af-4928-8816-7cf9b567e470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:42.846181 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db47f21-16af-4928-8816-7cf9b567e470 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:42.857578 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1426/2865] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:23:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:23:47.842013 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbc38b-856b-4443-b8f2-0cfad0ec63b1 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:47.842611 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbc38b-856b-4443-b8f2-0cfad0ec63b1 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:47.842611 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbc38b-856b-4443-b8f2-0cfad0ec63b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:47.874515 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-9fbbc38b-856b-4443-b8f2-0cfad0ec63b1 None None] MFA Rules not processed for user `53fd875fa50e4bee8bb7cece0efe4e9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:23:47.875741 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-9fbbc38b-856b-4443-b8f2-0cfad0ec63b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:47.906017 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:47.946011 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1440/2866] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:23:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:23:47.950692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db47f21-16af-4928-8816-7cf9b567e470 service neutron] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:47.960397 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db47f21-16af-4928-8816-7cf9b567e470 service neutron] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:47.965224 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d5dd9-ea46-49c4-8545-fb90de9b370e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:47.965845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d5dd9-ea46-49c4-8545-fb90de9b370e admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:47.966158 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d5dd9-ea46-49c4-8545-fb90de9b370e admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:47.966424 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d5dd9-ea46-49c4-8545-fb90de9b370e admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:47.967081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d5dd9-ea46-49c4-8545-fb90de9b370e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:47.968959 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d5dd9-ea46-49c4-8545-fb90de9b370e admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:48.000386 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1427/2867] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:23:47 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:23:48.004609 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbc38b-856b-4443-b8f2-0cfad0ec63b1 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:48.021194 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbc38b-856b-4443-b8f2-0cfad0ec63b1 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:48.084987 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c33284-658e-44ef-8c30-bdb5a2b2b020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:48.085644 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c33284-658e-44ef-8c30-bdb5a2b2b020 admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:48.085917 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c33284-658e-44ef-8c30-bdb5a2b2b020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:48.086185 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c33284-658e-44ef-8c30-bdb5a2b2b020 admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:48.086854 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c33284-658e-44ef-8c30-bdb5a2b2b020 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:48.088922 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c33284-658e-44ef-8c30-bdb5a2b2b020 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:48.112103 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1441/2868] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:23:48 2022] POST /identity/v3/users => generated 295 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:23:48.116187 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d5dd9-ea46-49c4-8545-fb90de9b370e admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:48.127206 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d5dd9-ea46-49c4-8545-fb90de9b370e admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:48.133198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecd195-cdd5-459d-bec0-486251934397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:48.133849 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecd195-cdd5-459d-bec0-486251934397 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:48.134126 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecd195-cdd5-459d-bec0-486251934397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:48.134393 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecd195-cdd5-459d-bec0-486251934397 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:48.135038 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecd195-cdd5-459d-bec0-486251934397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:48.136685 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecd195-cdd5-459d-bec0-486251934397 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:48.149799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1428/2869] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:23:48.154169 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c33284-658e-44ef-8c30-bdb5a2b2b020 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:48.169322 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c33284-658e-44ef-8c30-bdb5a2b2b020 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:48.174767 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e1a85e-c062-4c3b-96b6-2e245410668a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:48.175432 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1a85e-c062-4c3b-96b6-2e245410668a admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:48.175705 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1a85e-c062-4c3b-96b6-2e245410668a admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:48.176112 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1a85e-c062-4c3b-96b6-2e245410668a admin admin] PATH_INFO: `/v3/projects/73f7913c6fa84758b4ce8455d14ef3fe/users/5cb0dae982a44ca39f12e659e42bec8f/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:48.192939 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e1a85e-c062-4c3b-96b6-2e245410668a admin admin] RBAC: Authorizing `identity:create_grant(project_id=73f7913c6fa84758b4ce8455d14ef3fe, user_id=5cb0dae982a44ca39f12e659e42bec8f, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:48.195663 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e1a85e-c062-4c3b-96b6-2e245410668a admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:48.211894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1442/2870] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:23:48 2022] PUT /identity/v3/projects/73f7913c6fa84758b4ce8455d14ef3fe/users/5cb0dae982a44ca39f12e659e42bec8f/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:48.216965 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecd195-cdd5-459d-bec0-486251934397 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:48.227983 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecd195-cdd5-459d-bec0-486251934397 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:48.291379 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646e506-7071-468b-8669-4af34b8412fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:48.292018 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646e506-7071-468b-8669-4af34b8412fa admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:48.292272 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646e506-7071-468b-8669-4af34b8412fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:48.292539 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646e506-7071-468b-8669-4af34b8412fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:48.293196 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646e506-7071-468b-8669-4af34b8412fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:48.294904 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646e506-7071-468b-8669-4af34b8412fa admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:48.304942 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1429/2871] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:23:48.315063 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e1a85e-c062-4c3b-96b6-2e245410668a admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:48.335032 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e1a85e-c062-4c3b-96b6-2e245410668a admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:48.343296 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447aceaf-87ba-4e0c-8cb5-4f060da975d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:48.345738 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447aceaf-87ba-4e0c-8cb5-4f060da975d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:48.345738 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447aceaf-87ba-4e0c-8cb5-4f060da975d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:48.345738 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447aceaf-87ba-4e0c-8cb5-4f060da975d7 admin admin] PATH_INFO: `/v3/projects/73f7913c6fa84758b4ce8455d14ef3fe/users/5cb0dae982a44ca39f12e659e42bec8f/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:48.347777 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447aceaf-87ba-4e0c-8cb5-4f060da975d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73f7913c6fa84758b4ce8455d14ef3fe, user_id=5cb0dae982a44ca39f12e659e42bec8f, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:48.353114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447aceaf-87ba-4e0c-8cb5-4f060da975d7 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:48.371655 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1443/2872] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:23:48 2022] PUT /identity/v3/projects/73f7913c6fa84758b4ce8455d14ef3fe/users/5cb0dae982a44ca39f12e659e42bec8f/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:23:53.035920 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38454faa-52d4-47be-b267-5f292f7abab1 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:53.036412 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38454faa-52d4-47be-b267-5f292f7abab1 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:53.036679 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38454faa-52d4-47be-b267-5f292f7abab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:53.085390 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-38454faa-52d4-47be-b267-5f292f7abab1 None None] MFA Rules not processed for user `5cb0dae982a44ca39f12e659e42bec8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:23:53.086619 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-38454faa-52d4-47be-b267-5f292f7abab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:53.149021 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:53.182932 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1430/2873] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:23:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:23:53.196777 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447aceaf-87ba-4e0c-8cb5-4f060da975d7 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:53.198541 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-447aceaf-87ba-4e0c-8cb5-4f060da975d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:53.327406 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447aceaf-87ba-4e0c-8cb5-4f060da975d7 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:53.331343 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4f4ed-981b-46b6-a0a1-514b68f3ce75 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fe5293447346eaafd16818e5bd12f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:53.332044 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4f4ed-981b-46b6-a0a1-514b68f3ce75 service glance] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:53.332408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4f4ed-981b-46b6-a0a1-514b68f3ce75 service glance] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:53.332770 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4f4ed-981b-46b6-a0a1-514b68f3ce75 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:53.348247 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:23:53.399790 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4f4ed-981b-46b6-a0a1-514b68f3ce75 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:53.401286 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4f4ed-981b-46b6-a0a1-514b68f3ce75 service glance] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:53.411300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1444/2874] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:23:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:23:53.574176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755cfce-7f6c-4a3f-ae7f-5fa6bea5633f None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:53.574482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755cfce-7f6c-4a3f-ae7f-5fa6bea5633f None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:53.574893 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755cfce-7f6c-4a3f-ae7f-5fa6bea5633f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:53.644355 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-f755cfce-7f6c-4a3f-ae7f-5fa6bea5633f None None] MFA Rules not processed for user `3fada78ceaeb4f3bad77c516f6dfcfd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:23:53.645917 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-f755cfce-7f6c-4a3f-ae7f-5fa6bea5633f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:53.738989 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1431/2875] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 15:23:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:23:53.812932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4f4ed-981b-46b6-a0a1-514b68f3ce75 service glance] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:23:53.814762 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-11b4f4ed-981b-46b6-a0a1-514b68f3ce75 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:23:53.952800 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4f4ed-981b-46b6-a0a1-514b68f3ce75 service glance] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:23:53.958154 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c7675-9a1c-4011-adc6-4e14849c0dc6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05aa5e40ee4542719657bc03b2015e72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:23:53.958802 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c7675-9a1c-4011-adc6-4e14849c0dc6 service swift] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:23:53.959079 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c7675-9a1c-4011-adc6-4e14849c0dc6 service swift] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:23:53.959346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c7675-9a1c-4011-adc6-4e14849c0dc6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:23:53.968375 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c7675-9a1c-4011-adc6-4e14849c0dc6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:23:53.970018 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c7675-9a1c-4011-adc6-4e14849c0dc6 service swift] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:23:53.979284 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1445/2876] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 15:23:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 15:24:02.537842 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34088c-9861-4198-8596-21cb177e8210 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:24:02.538373 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34088c-9861-4198-8596-21cb177e8210 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:24:02.538373 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34088c-9861-4198-8596-21cb177e8210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:24:02.557176 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-3e34088c-9861-4198-8596-21cb177e8210 None None] MFA Rules not processed for user `53fd875fa50e4bee8bb7cece0efe4e9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:24:02.558381 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-3e34088c-9861-4198-8596-21cb177e8210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:24:02.565702 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1432/2877] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:24:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:24:02.615521 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c7675-9a1c-4011-adc6-4e14849c0dc6 service swift] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:24:02.627978 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c7675-9a1c-4011-adc6-4e14849c0dc6 service swift] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:24:02.633713 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b4a95-9556-4e04-8a85-f1b2d37ed686 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1fe5293447346eaafd16818e5bd12f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:24:02.634521 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b4a95-9556-4e04-8a85-f1b2d37ed686 service glance] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:24:02.634895 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b4a95-9556-4e04-8a85-f1b2d37ed686 service glance] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:24:02.635303 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b4a95-9556-4e04-8a85-f1b2d37ed686 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:24:02.637985 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-f76b4a95-9556-4e04-8a85-f1b2d37ed686 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:24:02.723691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b4a95-9556-4e04-8a85-f1b2d37ed686 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:24:02.725117 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b4a95-9556-4e04-8a85-f1b2d37ed686 service glance] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:24:02.736890 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1446/2878] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:24:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:24:03.003452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34088c-9861-4198-8596-21cb177e8210 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:24:03.066058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34088c-9861-4198-8596-21cb177e8210 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:24:03.069983 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ca65a-a808-4fcf-a691-f4befe7ba0ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:24:03.070591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ca65a-a808-4fcf-a691-f4befe7ba0ab service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:24:03.070862 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ca65a-a808-4fcf-a691-f4befe7ba0ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:24:03.071112 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ca65a-a808-4fcf-a691-f4befe7ba0ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:24:03.072969 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-903ca65a-a808-4fcf-a691-f4befe7ba0ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:24:03.083075 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903ca65a-a808-4fcf-a691-f4befe7ba0ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:24:03.084533 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903ca65a-a808-4fcf-a691-f4befe7ba0ab service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:24:03.093376 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1433/2879] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:24:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:24:03.326062 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b4a95-9556-4e04-8a85-f1b2d37ed686 service glance] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:24:03.335212 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b4a95-9556-4e04-8a85-f1b2d37ed686 service glance] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:24:03.338956 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be5731-d1ec-44bc-9c21-2b7c91afa882 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:24:03.339563 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be5731-d1ec-44bc-9c21-2b7c91afa882 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:24:03.339841 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be5731-d1ec-44bc-9c21-2b7c91afa882 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:24:03.340113 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be5731-d1ec-44bc-9c21-2b7c91afa882 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:24:03.341972 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-38be5731-d1ec-44bc-9c21-2b7c91afa882 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:24:03.355792 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be5731-d1ec-44bc-9c21-2b7c91afa882 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:24:03.357305 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be5731-d1ec-44bc-9c21-2b7c91afa882 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:24:03.362142 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=100 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:24:03.370181 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1447/2880] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:24:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:24:04.456585 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ca65a-a808-4fcf-a691-f4befe7ba0ab service neutron] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:24:04.458097 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-903ca65a-a808-4fcf-a691-f4befe7ba0ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:24:04.522238 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:24:04.543260 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ca65a-a808-4fcf-a691-f4befe7ba0ab service neutron] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:24:04.549870 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff00b72-221a-46a2-aff7-20823c21f6a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead06362f8a2444bbfe1405cd602f1c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:24:04.550419 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff00b72-221a-46a2-aff7-20823c21f6a1 service placement] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:24:04.550696 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff00b72-221a-46a2-aff7-20823c21f6a1 service placement] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:24:04.550989 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff00b72-221a-46a2-aff7-20823c21f6a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:24:04.552708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-bff00b72-221a-46a2-aff7-20823c21f6a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:24:04.567177 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff00b72-221a-46a2-aff7-20823c21f6a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:24:04.569401 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff00b72-221a-46a2-aff7-20823c21f6a1 service placement] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:24:04.580665 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1434/2881] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:24:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:24:08.976762 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826bc20-2b82-4f41-8a46-268dff7a8469 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:24:08.977313 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826bc20-2b82-4f41-8a46-268dff7a8469 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:24:08.977313 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826bc20-2b82-4f41-8a46-268dff7a8469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:24:08.997470 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-e826bc20-2b82-4f41-8a46-268dff7a8469 None None] MFA Rules not processed for user `3fada78ceaeb4f3bad77c516f6dfcfd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:24:08.998799 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-e826bc20-2b82-4f41-8a46-268dff7a8469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:24:09.006358 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1448/2882] 10.222.0.44 () {58 vars in 1009 bytes} [Sun Sep 11 15:24:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:24:09.020734 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff00b72-221a-46a2-aff7-20823c21f6a1 service placement] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:24:09.032767 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff00b72-221a-46a2-aff7-20823c21f6a1 service placement] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:24:09.041542 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ccb45-ca31-4348-8de4-4efb4e931685 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05aa5e40ee4542719657bc03b2015e72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:24:09.042342 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ccb45-ca31-4348-8de4-4efb4e931685 service swift] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:24:09.042677 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ccb45-ca31-4348-8de4-4efb4e931685 service swift] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:24:09.043009 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ccb45-ca31-4348-8de4-4efb4e931685 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:24:09.051672 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ccb45-ca31-4348-8de4-4efb4e931685 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:24:09.053345 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ccb45-ca31-4348-8de4-4efb4e931685 service swift] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:24:09.061881 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1435/2883] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 15:24:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 15:24:17.697589 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826bc20-2b82-4f41-8a46-268dff7a8469 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:24:17.699086 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-e826bc20-2b82-4f41-8a46-268dff7a8469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:24:17.715134 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826bc20-2b82-4f41-8a46-268dff7a8469 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:24:17.721180 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e873bb17-ec30-4fd9-8d6d-aa0e1a915e1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:24:17.721790 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873bb17-ec30-4fd9-8d6d-aa0e1a915e1b service nova] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:24:17.722322 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873bb17-ec30-4fd9-8d6d-aa0e1a915e1b service nova] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:24:17.722622 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873bb17-ec30-4fd9-8d6d-aa0e1a915e1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:24:17.724442 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-e873bb17-ec30-4fd9-8d6d-aa0e1a915e1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:24:17.738801 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e873bb17-ec30-4fd9-8d6d-aa0e1a915e1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:24:17.740664 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e873bb17-ec30-4fd9-8d6d-aa0e1a915e1b service nova] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:24:17.754055 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1449/2884] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:24:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:24:19.058310 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=14 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:24:20.647805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ccb45-ca31-4348-8de4-4efb4e931685 service swift] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:24:20.657880 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ccb45-ca31-4348-8de4-4efb4e931685 service swift] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:24:20.661954 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a9cfda-136a-4c85-80bf-62d14e7f57f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:24:20.662624 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9cfda-136a-4c85-80bf-62d14e7f57f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:24:20.662961 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9cfda-136a-4c85-80bf-62d14e7f57f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:24:20.663290 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9cfda-136a-4c85-80bf-62d14e7f57f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:24:20.665160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-d9a9cfda-136a-4c85-80bf-62d14e7f57f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:24:20.677120 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a9cfda-136a-4c85-80bf-62d14e7f57f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:24:20.678663 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a9cfda-136a-4c85-80bf-62d14e7f57f4 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:24:20.688223 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1436/2885] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:24:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:24:20.782266 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7b034-2d6e-48bd-9b3f-95e0fcecd7c6 None None] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:24:20.782812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7b034-2d6e-48bd-9b3f-95e0fcecd7c6 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:24:20.782812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7b034-2d6e-48bd-9b3f-95e0fcecd7c6 None None] PATH_INFO: `/` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:24:20.784134 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1450/2886] 10.222.0.44 () {54 vars in 921 bytes} [Sun Sep 11 15:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 15:24:20.790008 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b6900-d3f7-4311-bf5b-d569fdcb029d None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:24:20.790287 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b6900-d3f7-4311-bf5b-d569fdcb029d None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:24:20.790551 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b6900-d3f7-4311-bf5b-d569fdcb029d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:24:20.837865 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-ff7b6900-d3f7-4311-bf5b-d569fdcb029d None None] MFA Rules not processed for user `453297dac64046d19fc1c331d2d85abc`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:24:20.839058 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-ff7b6900-d3f7-4311-bf5b-d569fdcb029d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:24:20.848775 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1437/2887] 10.222.0.44 () {58 vars in 1018 bytes} [Sun Sep 11 15:24:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:24:20.860152 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7b034-2d6e-48bd-9b3f-95e0fcecd7c6 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:24:20.863170 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-edd7b034-2d6e-48bd-9b3f-95e0fcecd7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:24:20.881064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7b034-2d6e-48bd-9b3f-95e0fcecd7c6 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:24:20.884903 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69324ac9-548f-4497-8e7a-4b2a26e6f4fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:24:20.885545 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69324ac9-548f-4497-8e7a-4b2a26e6f4fe service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:24:20.885820 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69324ac9-548f-4497-8e7a-4b2a26e6f4fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:24:20.886083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69324ac9-548f-4497-8e7a-4b2a26e6f4fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:24:20.896365 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69324ac9-548f-4497-8e7a-4b2a26e6f4fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:24:20.897809 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69324ac9-548f-4497-8e7a-4b2a26e6f4fe service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:24:20.909649 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1451/2888] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:24:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:24:23.017308 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b6900-d3f7-4311-bf5b-d569fdcb029d None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:24:23.018767 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-ff7b6900-d3f7-4311-bf5b-d569fdcb029d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:24:23.034258 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b6900-d3f7-4311-bf5b-d569fdcb029d None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:24:23.039980 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bbdc29-04b5-4d87-adfb-718f87741573 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:24:23.040612 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bbdc29-04b5-4d87-adfb-718f87741573 service nova] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:24:23.040908 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bbdc29-04b5-4d87-adfb-718f87741573 service nova] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:24:23.041201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bbdc29-04b5-4d87-adfb-718f87741573 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:24:23.044619 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-41bbdc29-04b5-4d87-adfb-718f87741573 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:24:23.059222 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bbdc29-04b5-4d87-adfb-718f87741573 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:24:23.061167 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bbdc29-04b5-4d87-adfb-718f87741573 service nova] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:24:23.072654 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1438/2889] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:24:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:24:30.903851 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=16 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:24:33.068708 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=21 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:24:37.534279 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69324ac9-548f-4497-8e7a-4b2a26e6f4fe service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:24:37.546610 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69324ac9-548f-4497-8e7a-4b2a26e6f4fe service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:24:37.551192 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920041a1-9cba-44ab-b8fb-66e4e651dcfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:24:37.551464 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920041a1-9cba-44ab-b8fb-66e4e651dcfd service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:24:37.551739 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920041a1-9cba-44ab-b8fb-66e4e651dcfd service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:24:37.552005 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920041a1-9cba-44ab-b8fb-66e4e651dcfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:24:37.569187 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920041a1-9cba-44ab-b8fb-66e4e651dcfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:24:37.572040 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920041a1-9cba-44ab-b8fb-66e4e651dcfd service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:24:37.582517 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1452/2890] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:24:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:24:41.348698 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bbdc29-04b5-4d87-adfb-718f87741573 service nova] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:24:41.362564 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bbdc29-04b5-4d87-adfb-718f87741573 service nova] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:24:41.374457 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce002783-0b76-4e06-982c-e8c69cfe05fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:24:41.375296 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce002783-0b76-4e06-982c-e8c69cfe05fe service nova] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:24:41.375716 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce002783-0b76-4e06-982c-e8c69cfe05fe service nova] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:24:41.375992 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce002783-0b76-4e06-982c-e8c69cfe05fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:24:41.378793 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-ce002783-0b76-4e06-982c-e8c69cfe05fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:24:41.394699 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce002783-0b76-4e06-982c-e8c69cfe05fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:24:41.398943 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce002783-0b76-4e06-982c-e8c69cfe05fe service nova] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:24:41.412462 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1439/2891] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:24:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:24:47.576879 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=6 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:24:51.408572 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=6 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:25:58.654917 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920041a1-9cba-44ab-b8fb-66e4e651dcfd service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:25:58.657158 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-920041a1-9cba-44ab-b8fb-66e4e651dcfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:25:58.677204 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920041a1-9cba-44ab-b8fb-66e4e651dcfd service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:25:58.683832 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12871b89-3548-497d-8018-eb910181ef79 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead06362f8a2444bbfe1405cd602f1c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:25:58.684451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12871b89-3548-497d-8018-eb910181ef79 service placement] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:25:58.684718 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12871b89-3548-497d-8018-eb910181ef79 service placement] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:25:58.684980 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12871b89-3548-497d-8018-eb910181ef79 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:25:58.687330 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-12871b89-3548-497d-8018-eb910181ef79 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:25:58.702673 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12871b89-3548-497d-8018-eb910181ef79 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:25:58.704474 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12871b89-3548-497d-8018-eb910181ef79 service placement] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:25:58.715998 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce002783-0b76-4e06-982c-e8c69cfe05fe service nova] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:25:58.718330 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-ce002783-0b76-4e06-982c-e8c69cfe05fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:25:58.718847 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1453/2892] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:25:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:25:58.800725 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce002783-0b76-4e06-982c-e8c69cfe05fe service nova] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:25:58.805656 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b4c05-1099-4581-b8a6-6ebe3d4aaaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:25:58.806295 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b4c05-1099-4581-b8a6-6ebe3d4aaaec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:25:58.806569 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b4c05-1099-4581-b8a6-6ebe3d4aaaec admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:25:58.806840 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b4c05-1099-4581-b8a6-6ebe3d4aaaec admin admin] PATH_INFO: `/v3/users/6e253dad57f44ab2955c353792fd5097` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:25:58.808012 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26b4c05-1099-4581-b8a6-6ebe3d4aaaec admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e253dad57f44ab2955c353792fd5097)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:25:58.810030 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26b4c05-1099-4581-b8a6-6ebe3d4aaaec admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:25:58.859434 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-b26b4c05-1099-4581-b8a6-6ebe3d4aaaec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e253dad57f44ab2955c353792fd5097'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:25:58.868544 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-b26b4c05-1099-4581-b8a6-6ebe3d4aaaec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e253dad57f44ab2955c353792fd5097'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:25:58.880452 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-b26b4c05-1099-4581-b8a6-6ebe3d4aaaec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e253dad57f44ab2955c353792fd5097'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:25:58.881079 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-b26b4c05-1099-4581-b8a6-6ebe3d4aaaec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e253dad57f44ab2955c353792fd5097'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:25:58.888000 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-b26b4c05-1099-4581-b8a6-6ebe3d4aaaec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e253dad57f44ab2955c353792fd5097'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:25:58.893895 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1440/2893] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:25:58 2022] DELETE /identity/v3/users/6e253dad57f44ab2955c353792fd5097 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:25:59.071090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12871b89-3548-497d-8018-eb910181ef79 service placement] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:25:59.084031 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12871b89-3548-497d-8018-eb910181ef79 service placement] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:25:59.152769 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29424a-7a1f-44bb-a996-4424dd80b711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:25:59.153520 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29424a-7a1f-44bb-a996-4424dd80b711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:25:59.153966 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29424a-7a1f-44bb-a996-4424dd80b711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:25:59.154407 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29424a-7a1f-44bb-a996-4424dd80b711 admin admin] PATH_INFO: `/v3/projects/ae3105c3aa57472899713ef172a862e4` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:25:59.155861 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea29424a-7a1f-44bb-a996-4424dd80b711 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae3105c3aa57472899713ef172a862e4)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:25:59.159182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea29424a-7a1f-44bb-a996-4424dd80b711 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:25:59.214078 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-ea29424a-7a1f-44bb-a996-4424dd80b711 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae3105c3aa57472899713ef172a862e4', 'request_id': 'req-ea29424a-7a1f-44bb-a996-4424dd80b711'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:25:59.220602 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-ea29424a-7a1f-44bb-a996-4424dd80b711 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae3105c3aa57472899713ef172a862e4', 'request_id': 'req-ea29424a-7a1f-44bb-a996-4424dd80b711'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:25:59.227235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-ea29424a-7a1f-44bb-a996-4424dd80b711 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae3105c3aa57472899713ef172a862e4', 'request_id': 'req-ea29424a-7a1f-44bb-a996-4424dd80b711'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:25:59.241804 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-ea29424a-7a1f-44bb-a996-4424dd80b711 admin admin] The token cache is being invalidate because project ae3105c3aa57472899713ef172a862e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:25:59.242090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-ea29424a-7a1f-44bb-a996-4424dd80b711 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:25:59.242758 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-ea29424a-7a1f-44bb-a996-4424dd80b711 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:25:59.244313 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1454/2894] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:25:59 2022] DELETE /identity/v3/projects/ae3105c3aa57472899713ef172a862e4 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:25:59.248803 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b4c05-1099-4581-b8a6-6ebe3d4aaaec admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:25:59.250244 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b26b4c05-1099-4581-b8a6-6ebe3d4aaaec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:25:59.263620 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b4c05-1099-4581-b8a6-6ebe3d4aaaec admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:25:59.331590 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629228d-f39e-4a4b-8836-17fc97ec227a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:25:59.332464 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629228d-f39e-4a4b-8836-17fc97ec227a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:25:59.332819 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629228d-f39e-4a4b-8836-17fc97ec227a admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:25:59.333394 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629228d-f39e-4a4b-8836-17fc97ec227a admin admin] PATH_INFO: `/v3/users/cf0ef0a9c79d4b4b83ec78f51e304972` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:25:59.335647 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3629228d-f39e-4a4b-8836-17fc97ec227a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf0ef0a9c79d4b4b83ec78f51e304972)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:25:59.337725 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3629228d-f39e-4a4b-8836-17fc97ec227a admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:25:59.365690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:25:59.396244 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3629228d-f39e-4a4b-8836-17fc97ec227a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf0ef0a9c79d4b4b83ec78f51e304972'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:25:59.406238 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3629228d-f39e-4a4b-8836-17fc97ec227a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf0ef0a9c79d4b4b83ec78f51e304972'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:25:59.418651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3629228d-f39e-4a4b-8836-17fc97ec227a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf0ef0a9c79d4b4b83ec78f51e304972'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:25:59.419311 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3629228d-f39e-4a4b-8836-17fc97ec227a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf0ef0a9c79d4b4b83ec78f51e304972'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:25:59.426458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-3629228d-f39e-4a4b-8836-17fc97ec227a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf0ef0a9c79d4b4b83ec78f51e304972'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:25:59.432453 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1441/2895] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:25:59 2022] DELETE /identity/v3/users/cf0ef0a9c79d4b4b83ec78f51e304972 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:25:59.609651 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29424a-7a1f-44bb-a996-4424dd80b711 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:25:59.611390 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-ea29424a-7a1f-44bb-a996-4424dd80b711 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:25:59.625964 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29424a-7a1f-44bb-a996-4424dd80b711 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:25:59.684955 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297fa2bc-b0b2-4cb2-9e32-7979de61193e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:25:59.686146 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297fa2bc-b0b2-4cb2-9e32-7979de61193e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:25:59.686257 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297fa2bc-b0b2-4cb2-9e32-7979de61193e admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:25:59.686483 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297fa2bc-b0b2-4cb2-9e32-7979de61193e admin admin] PATH_INFO: `/v3/projects/864272140621484fbb0ffa59274d101e` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:25:59.687619 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297fa2bc-b0b2-4cb2-9e32-7979de61193e admin admin] RBAC: Authorizing `identity:delete_project(project_id=864272140621484fbb0ffa59274d101e)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:25:59.689546 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297fa2bc-b0b2-4cb2-9e32-7979de61193e admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:25:59.711773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:25:59.740054 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-297fa2bc-b0b2-4cb2-9e32-7979de61193e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '864272140621484fbb0ffa59274d101e', 'request_id': 'req-297fa2bc-b0b2-4cb2-9e32-7979de61193e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:25:59.746034 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-297fa2bc-b0b2-4cb2-9e32-7979de61193e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '864272140621484fbb0ffa59274d101e', 'request_id': 'req-297fa2bc-b0b2-4cb2-9e32-7979de61193e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:25:59.752322 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-297fa2bc-b0b2-4cb2-9e32-7979de61193e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '864272140621484fbb0ffa59274d101e', 'request_id': 'req-297fa2bc-b0b2-4cb2-9e32-7979de61193e'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:25:59.767386 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-297fa2bc-b0b2-4cb2-9e32-7979de61193e admin admin] The token cache is being invalidate because project 864272140621484fbb0ffa59274d101e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:25:59.767673 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-297fa2bc-b0b2-4cb2-9e32-7979de61193e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:25:59.768388 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-297fa2bc-b0b2-4cb2-9e32-7979de61193e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:25:59.770024 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1455/2896] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:25:59 2022] DELETE /identity/v3/projects/864272140621484fbb0ffa59274d101e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:25:59.775221 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f8bfc-7770-4dc0-a51e-e20166684b65 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:25:59.775754 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f8bfc-7770-4dc0-a51e-e20166684b65 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:25:59.775754 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f8bfc-7770-4dc0-a51e-e20166684b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:25:59.796750 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-6d9f8bfc-7770-4dc0-a51e-e20166684b65 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:25:59.797955 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-6d9f8bfc-7770-4dc0-a51e-e20166684b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:25:59.859169 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1442/2897] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:25:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:25:59.868411 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef9d75-4335-472e-9e06-4a77a4ed642e None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:25:59.869344 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef9d75-4335-472e-9e06-4a77a4ed642e None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:25:59.869678 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef9d75-4335-472e-9e06-4a77a4ed642e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:25:59.898288 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-7cef9d75-4335-472e-9e06-4a77a4ed642e None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:25:59.899516 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-7cef9d75-4335-472e-9e06-4a77a4ed642e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:25:59.908268 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1456/2898] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:25:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:25:59.912690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f8bfc-7770-4dc0-a51e-e20166684b65 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:25:59.922681 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f8bfc-7770-4dc0-a51e-e20166684b65 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:25:59.927675 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1400a7d-caf4-41b2-90a4-825bf7e81e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:25:59.928266 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1400a7d-caf4-41b2-90a4-825bf7e81e69 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:25:59.928537 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1400a7d-caf4-41b2-90a4-825bf7e81e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:25:59.928805 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1400a7d-caf4-41b2-90a4-825bf7e81e69 admin admin] PATH_INFO: `/v3/domains` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:25:59.929511 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1400a7d-caf4-41b2-90a4-825bf7e81e69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81684) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 15:25:59.929799 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1400a7d-caf4-41b2-90a4-825bf7e81e69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:25:59.931406 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1400a7d-caf4-41b2-90a4-825bf7e81e69 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:25:59.943730 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1443/2899] 10.222.0.44 () {58 vars in 1153 bytes} [Sun Sep 11 15:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 15:25:59.947935 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef9d75-4335-472e-9e06-4a77a4ed642e None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:25:59.957558 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef9d75-4335-472e-9e06-4a77a4ed642e None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:25:59.962172 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1010b7e8-e157-4795-86ff-99d8e0a3ce2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:25:59.962780 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010b7e8-e157-4795-86ff-99d8e0a3ce2b admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:25:59.963053 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010b7e8-e157-4795-86ff-99d8e0a3ce2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:25:59.963344 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010b7e8-e157-4795-86ff-99d8e0a3ce2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:25:59.963955 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1010b7e8-e157-4795-86ff-99d8e0a3ce2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:25:59.965889 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1010b7e8-e157-4795-86ff-99d8e0a3ce2b admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:25:59.993961 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1457/2900] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:25:59 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:25:59.998381 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1400a7d-caf4-41b2-90a4-825bf7e81e69 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:26:00.008256 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1400a7d-caf4-41b2-90a4-825bf7e81e69 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:26:00.068783 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5b3aa-50f2-4e0d-919c-fbf76a68aefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:26:00.069429 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5b3aa-50f2-4e0d-919c-fbf76a68aefb admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:26:00.069721 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5b3aa-50f2-4e0d-919c-fbf76a68aefb admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:26:00.069987 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5b3aa-50f2-4e0d-919c-fbf76a68aefb admin admin] PATH_INFO: `/v3/users` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:26:00.071427 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f5b3aa-50f2-4e0d-919c-fbf76a68aefb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:26:00.072738 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f5b3aa-50f2-4e0d-919c-fbf76a68aefb admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:26:00.094855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1444/2901] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:25:59 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:26:00.099071 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1010b7e8-e157-4795-86ff-99d8e0a3ce2b admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:26:00.110078 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1010b7e8-e157-4795-86ff-99d8e0a3ce2b admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:26:00.115451 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c56095-28c7-4493-90be-34f615a20ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:26:00.116079 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c56095-28c7-4493-90be-34f615a20ea9 admin admin] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:26:00.116347 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c56095-28c7-4493-90be-34f615a20ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:26:00.116725 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c56095-28c7-4493-90be-34f615a20ea9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:26:00.117456 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c56095-28c7-4493-90be-34f615a20ea9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:26:00.119183 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c56095-28c7-4493-90be-34f615a20ea9 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:26:00.130488 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1458/2902] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 15:26:00.134821 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5b3aa-50f2-4e0d-919c-fbf76a68aefb admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:26:00.145534 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5b3aa-50f2-4e0d-919c-fbf76a68aefb admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:26:00.150515 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7caa29f-5ae7-43e3-b3a2-4da2e94cc85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:26:00.151129 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7caa29f-5ae7-43e3-b3a2-4da2e94cc85a admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:26:00.151458 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7caa29f-5ae7-43e3-b3a2-4da2e94cc85a admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:26:00.151722 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7caa29f-5ae7-43e3-b3a2-4da2e94cc85a admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:26:00.152363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7caa29f-5ae7-43e3-b3a2-4da2e94cc85a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:26:00.154005 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7caa29f-5ae7-43e3-b3a2-4da2e94cc85a admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:26:00.165705 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1445/2903] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:26:00.169724 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c56095-28c7-4493-90be-34f615a20ea9 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:26:00.180025 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c56095-28c7-4493-90be-34f615a20ea9 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:26:00.189179 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b0d38-4cea-453d-84a1-d1d00b99bba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:26:00.190083 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b0d38-4cea-453d-84a1-d1d00b99bba9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:26:00.190518 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b0d38-4cea-453d-84a1-d1d00b99bba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:26:00.190937 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b0d38-4cea-453d-84a1-d1d00b99bba9 admin admin] PATH_INFO: `/v3/projects/521053acc38643e09eb22ead90cd2ba1/users/0a70540aee424f3896e43e4d6af5c63a/roles/3e3756d9482f4b9b86271eacdc19f523` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:26:00.210787 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b0d38-4cea-453d-84a1-d1d00b99bba9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=521053acc38643e09eb22ead90cd2ba1, user_id=0a70540aee424f3896e43e4d6af5c63a, role_id=3e3756d9482f4b9b86271eacdc19f523)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:26:00.213439 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b0d38-4cea-453d-84a1-d1d00b99bba9 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:26:00.228900 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1459/2904] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:26:00 2022] PUT /identity/v3/projects/521053acc38643e09eb22ead90cd2ba1/users/0a70540aee424f3896e43e4d6af5c63a/roles/3e3756d9482f4b9b86271eacdc19f523 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:26:00.234160 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa30c2-90d1-47d6-a306-00d87a2f7d88 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:26:00.234587 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa30c2-90d1-47d6-a306-00d87a2f7d88 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:26:00.234860 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa30c2-90d1-47d6-a306-00d87a2f7d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:26:00.250540 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:26:00.269120 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-c0fa30c2-90d1-47d6-a306-00d87a2f7d88 None None] MFA Rules not processed for user `e6c6442222c54f34a84fdb4577045a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:26:00.271236 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c0fa30c2-90d1-47d6-a306-00d87a2f7d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:26:00.346008 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1446/2905] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:26:00.355789 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b0d38-4cea-453d-84a1-d1d00b99bba9 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:26:00.357598 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-2d2b0d38-4cea-453d-84a1-d1d00b99bba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:26:00.378546 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa30c2-90d1-47d6-a306-00d87a2f7d88 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:26:00.380142 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c0fa30c2-90d1-47d6-a306-00d87a2f7d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:26:00.419070 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b0d38-4cea-453d-84a1-d1d00b99bba9 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:26:00.422948 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a266cf76-9d03-43cb-872b-a45d3d9f77bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:26:00.423636 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266cf76-9d03-43cb-872b-a45d3d9f77bb service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:26:00.424050 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266cf76-9d03-43cb-872b-a45d3d9f77bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:26:00.424343 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266cf76-9d03-43cb-872b-a45d3d9f77bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:26:00.433787 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a266cf76-9d03-43cb-872b-a45d3d9f77bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:26:00.435215 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a266cf76-9d03-43cb-872b-a45d3d9f77bb service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:26:00.446274 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1460/2906] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:26:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:26:00.480328 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa30c2-90d1-47d6-a306-00d87a2f7d88 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:26:00.485730 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff2249-c8c8-4e1f-b96d-891085d998f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead06362f8a2444bbfe1405cd602f1c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:26:00.486428 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff2249-c8c8-4e1f-b96d-891085d998f3 service placement] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:26:00.486769 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff2249-c8c8-4e1f-b96d-891085d998f3 service placement] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:26:00.487106 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff2249-c8c8-4e1f-b96d-891085d998f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:26:00.489041 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-9aff2249-c8c8-4e1f-b96d-891085d998f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:26:00.502531 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff2249-c8c8-4e1f-b96d-891085d998f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:26:00.504259 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff2249-c8c8-4e1f-b96d-891085d998f3 service placement] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:26:00.517344 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1447/2907] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:26:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:26:01.647479 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3d29d-4756-4260-92fa-bc8020ccf59c None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:26:01.648324 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3d29d-4756-4260-92fa-bc8020ccf59c None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:26:01.648785 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3d29d-4756-4260-92fa-bc8020ccf59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:26:01.669258 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-32a3d29d-4756-4260-92fa-bc8020ccf59c None None] MFA Rules not processed for user `3fada78ceaeb4f3bad77c516f6dfcfd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:26:01.670547 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-32a3d29d-4756-4260-92fa-bc8020ccf59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:26:01.718388 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:26:01.739078 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1461/2908] 10.222.0.44 () {58 vars in 967 bytes} [Sun Sep 11 15:26:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:26:01.801741 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff2249-c8c8-4e1f-b96d-891085d998f3 service placement] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:26:01.803363 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-9aff2249-c8c8-4e1f-b96d-891085d998f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:26:01.883812 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff2249-c8c8-4e1f-b96d-891085d998f3 service placement] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:26:01.890750 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737289e-6123-4bc7-b211-cb08aacf32f4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05aa5e40ee4542719657bc03b2015e72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:26:01.891454 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737289e-6123-4bc7-b211-cb08aacf32f4 service swift] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:26:01.891795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737289e-6123-4bc7-b211-cb08aacf32f4 service swift] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:26:01.892127 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737289e-6123-4bc7-b211-cb08aacf32f4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:26:01.903920 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4737289e-6123-4bc7-b211-cb08aacf32f4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:26:01.906066 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4737289e-6123-4bc7-b211-cb08aacf32f4 service swift] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:26:01.915331 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1448/2909] 10.222.0.44 () {60 vars in 1384 bytes} [Sun Sep 11 15:26:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 15:26:04.999778 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3e3d6-ef1a-4cfe-99f0-b4c7bfff272a None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:26:05.000396 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3e3d6-ef1a-4cfe-99f0-b4c7bfff272a None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:26:05.000396 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3e3d6-ef1a-4cfe-99f0-b4c7bfff272a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:26:05.037289 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-36f3e3d6-ef1a-4cfe-99f0-b4c7bfff272a None None] MFA Rules not processed for user `0a70540aee424f3896e43e4d6af5c63a`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:26:05.039161 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-36f3e3d6-ef1a-4cfe-99f0-b4c7bfff272a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:26:05.138090 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1462/2910] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:26:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:26:05.139351 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737289e-6123-4bc7-b211-cb08aacf32f4 service swift] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:26:05.152151 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737289e-6123-4bc7-b211-cb08aacf32f4 service swift] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:26:05.158551 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b329f-071a-410d-9aa9-95f11e48501e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:26:05.159356 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b329f-071a-410d-9aa9-95f11e48501e admin admin] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:26:05.159774 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b329f-071a-410d-9aa9-95f11e48501e admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:26:05.160169 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b329f-071a-410d-9aa9-95f11e48501e admin admin] PATH_INFO: `/v3/projects` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:26:05.160907 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b329f-071a-410d-9aa9-95f11e48501e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:26:05.164092 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b329f-071a-410d-9aa9-95f11e48501e admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:26:05.210038 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1449/2911] 10.222.0.44 () {60 vars in 1153 bytes} [Sun Sep 11 15:26:05 2022] POST /identity/v3/projects => generated 377 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 15:26:05.213772 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f3e3d6-ef1a-4cfe-99f0-b4c7bfff272a None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:26:05.228812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f3e3d6-ef1a-4cfe-99f0-b4c7bfff272a None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:26:05.343974 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ba01d-b0c8-4439-bc9c-56b94976f98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:26:05.344575 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ba01d-b0c8-4439-bc9c-56b94976f98f admin admin] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:26:05.344973 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ba01d-b0c8-4439-bc9c-56b94976f98f admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:26:05.345410 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ba01d-b0c8-4439-bc9c-56b94976f98f admin admin] PATH_INFO: `/v3/users` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:26:05.346051 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ba01d-b0c8-4439-bc9c-56b94976f98f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:26:05.348046 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ba01d-b0c8-4439-bc9c-56b94976f98f admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:26:05.373986 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1463/2912] 10.222.0.44 () {60 vars in 1144 bytes} [Sun Sep 11 15:26:05 2022] POST /identity/v3/users => generated 295 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 15:26:05.378022 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b329f-071a-410d-9aa9-95f11e48501e admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:26:05.383761 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:26:05.390558 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b329f-071a-410d-9aa9-95f11e48501e admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:26:05.396201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b733b-fe70-4d4b-8f07-05f466144e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:26:05.396855 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b733b-fe70-4d4b-8f07-05f466144e88 admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:26:05.397172 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b733b-fe70-4d4b-8f07-05f466144e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:26:05.397443 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b733b-fe70-4d4b-8f07-05f466144e88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:26:05.398264 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b733b-fe70-4d4b-8f07-05f466144e88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:26:05.399913 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b733b-fe70-4d4b-8f07-05f466144e88 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:26:05.413781 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1450/2913] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:26:05.418002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ba01d-b0c8-4439-bc9c-56b94976f98f admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:26:05.429002 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ba01d-b0c8-4439-bc9c-56b94976f98f admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:26:05.433934 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bffe77-fbe5-42fe-9c3a-8080df72e2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:26:05.434850 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bffe77-fbe5-42fe-9c3a-8080df72e2ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:26:05.435202 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bffe77-fbe5-42fe-9c3a-8080df72e2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:26:05.436023 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bffe77-fbe5-42fe-9c3a-8080df72e2ba admin admin] PATH_INFO: `/v3/projects/e89b26aada41478396b13c82a270ab56/users/9689c4018b334a9380f269772758e2f5/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:26:05.452996 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bffe77-fbe5-42fe-9c3a-8080df72e2ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=e89b26aada41478396b13c82a270ab56, user_id=9689c4018b334a9380f269772758e2f5, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:26:05.455757 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bffe77-fbe5-42fe-9c3a-8080df72e2ba admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:26:05.470201 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1464/2914] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:26:05 2022] PUT /identity/v3/projects/e89b26aada41478396b13c82a270ab56/users/9689c4018b334a9380f269772758e2f5/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:26:05.474273 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b733b-fe70-4d4b-8f07-05f466144e88 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:26:05.484426 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b733b-fe70-4d4b-8f07-05f466144e88 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:26:05.543296 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c794c-792d-472c-aa88-4e034312e50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:26:05.543942 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c794c-792d-472c-aa88-4e034312e50b admin admin] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:26:05.544214 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c794c-792d-472c-aa88-4e034312e50b admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:26:05.544480 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c794c-792d-472c-aa88-4e034312e50b admin admin] PATH_INFO: `/v3/roles` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:26:05.545123 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c794c-792d-472c-aa88-4e034312e50b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:26:05.546841 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c794c-792d-472c-aa88-4e034312e50b admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:26:05.556935 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1451/2915] 10.222.0.44 () {58 vars in 1122 bytes} [Sun Sep 11 15:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 15:26:05.561641 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bffe77-fbe5-42fe-9c3a-8080df72e2ba admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:26:05.573551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bffe77-fbe5-42fe-9c3a-8080df72e2ba admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:26:05.578581 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652524e0-2ccd-4508-851c-05ea9fc16025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:26:05.578955 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652524e0-2ccd-4508-851c-05ea9fc16025 admin admin] REQUEST_METHOD: `PUT` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:26:05.579267 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652524e0-2ccd-4508-851c-05ea9fc16025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:26:05.579593 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652524e0-2ccd-4508-851c-05ea9fc16025 admin admin] PATH_INFO: `/v3/projects/e89b26aada41478396b13c82a270ab56/users/9689c4018b334a9380f269772758e2f5/roles/c426875a15834aa5b122e2a62267d15a` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:26:05.581499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652524e0-2ccd-4508-851c-05ea9fc16025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e89b26aada41478396b13c82a270ab56, user_id=9689c4018b334a9380f269772758e2f5, role_id=c426875a15834aa5b122e2a62267d15a)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:26:05.584326 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652524e0-2ccd-4508-851c-05ea9fc16025 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:26:05.596037 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1465/2916] 10.222.0.44 () {60 vars in 1483 bytes} [Sun Sep 11 15:26:05 2022] PUT /identity/v3/projects/e89b26aada41478396b13c82a270ab56/users/9689c4018b334a9380f269772758e2f5/roles/c426875a15834aa5b122e2a62267d15a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 15:26:11.036146 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c794c-792d-472c-aa88-4e034312e50b admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:26:11.038165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-348c794c-792d-472c-aa88-4e034312e50b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:26:11.061493 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c794c-792d-472c-aa88-4e034312e50b admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:26:11.143695 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35a4ca-28cc-437a-8738-91d788574d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:26:11.143695 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35a4ca-28cc-437a-8738-91d788574d97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:26:11.143695 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35a4ca-28cc-437a-8738-91d788574d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:26:11.143695 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35a4ca-28cc-437a-8738-91d788574d97 admin admin] PATH_INFO: `/v3/users/53fd875fa50e4bee8bb7cece0efe4e9b` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:26:11.144428 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b35a4ca-28cc-437a-8738-91d788574d97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53fd875fa50e4bee8bb7cece0efe4e9b)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:26:11.144428 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b35a4ca-28cc-437a-8738-91d788574d97 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:26:11.211126 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-9b35a4ca-28cc-437a-8738-91d788574d97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53fd875fa50e4bee8bb7cece0efe4e9b'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:26:11.224370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-9b35a4ca-28cc-437a-8738-91d788574d97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53fd875fa50e4bee8bb7cece0efe4e9b'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:26:11.240846 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-9b35a4ca-28cc-437a-8738-91d788574d97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53fd875fa50e4bee8bb7cece0efe4e9b'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:26:11.244460 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-9b35a4ca-28cc-437a-8738-91d788574d97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53fd875fa50e4bee8bb7cece0efe4e9b'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:26:11.257099 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-9b35a4ca-28cc-437a-8738-91d788574d97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53fd875fa50e4bee8bb7cece0efe4e9b'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:26:11.264051 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1452/2917] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:26:11 2022] DELETE /identity/v3/users/53fd875fa50e4bee8bb7cece0efe4e9b => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:26:11.517346 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652524e0-2ccd-4508-851c-05ea9fc16025 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:26:11.519210 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-652524e0-2ccd-4508-851c-05ea9fc16025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:26:11.540318 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652524e0-2ccd-4508-851c-05ea9fc16025 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:26:11.616499 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:26:11.684937 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b727176-04fe-4b44-a4af-006ef92f8a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:26:11.684937 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b727176-04fe-4b44-a4af-006ef92f8a44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:26:11.684937 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b727176-04fe-4b44-a4af-006ef92f8a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:26:11.684937 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b727176-04fe-4b44-a4af-006ef92f8a44 admin admin] PATH_INFO: `/v3/projects/5b8c0728fc2e4e54ab90bbce9be73862` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:26:11.685722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b727176-04fe-4b44-a4af-006ef92f8a44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b8c0728fc2e4e54ab90bbce9be73862)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:26:11.685722 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b727176-04fe-4b44-a4af-006ef92f8a44 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:26:11.687010 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b00b6-db31-4718-8ba1-b64c25f722b0 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:26:11.687303 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b00b6-db31-4718-8ba1-b64c25f722b0 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:26:11.687575 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b00b6-db31-4718-8ba1-b64c25f722b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:26:11.704214 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:26:11.735444 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-c96b00b6-db31-4718-8ba1-b64c25f722b0 None None] MFA Rules not processed for user `9689c4018b334a9380f269772758e2f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:26:11.737091 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c96b00b6-db31-4718-8ba1-b64c25f722b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:26:11.768812 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-7b727176-04fe-4b44-a4af-006ef92f8a44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b8c0728fc2e4e54ab90bbce9be73862', 'request_id': 'req-7b727176-04fe-4b44-a4af-006ef92f8a44'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:26:11.775629 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-7b727176-04fe-4b44-a4af-006ef92f8a44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b8c0728fc2e4e54ab90bbce9be73862', 'request_id': 'req-7b727176-04fe-4b44-a4af-006ef92f8a44'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:26:11.790720 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-7b727176-04fe-4b44-a4af-006ef92f8a44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b8c0728fc2e4e54ab90bbce9be73862', 'request_id': 'req-7b727176-04fe-4b44-a4af-006ef92f8a44'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:26:11.814507 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-7b727176-04fe-4b44-a4af-006ef92f8a44 admin admin] The token cache is being invalidate because project 5b8c0728fc2e4e54ab90bbce9be73862 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:26:11.814918 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-7b727176-04fe-4b44-a4af-006ef92f8a44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:26:11.815594 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-7b727176-04fe-4b44-a4af-006ef92f8a44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:26:11.822894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1466/2918] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:26:11 2022] DELETE /identity/v3/projects/5b8c0728fc2e4e54ab90bbce9be73862 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:26:11.824182 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b727176-04fe-4b44-a4af-006ef92f8a44 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:26:11.826424 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-7b727176-04fe-4b44-a4af-006ef92f8a44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:26:11.843921 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1453/2919] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:26:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 15:26:11.858036 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b00b6-db31-4718-8ba1-b64c25f722b0 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:26:11.860306 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-c96b00b6-db31-4718-8ba1-b64c25f722b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:26:11.945128 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b727176-04fe-4b44-a4af-006ef92f8a44 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:26:11.950746 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fb602-24f0-4131-be9e-b9256a717ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:26:11.951590 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fb602-24f0-4131-be9e-b9256a717ab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:26:11.951961 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fb602-24f0-4131-be9e-b9256a717ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:26:11.952353 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fb602-24f0-4131-be9e-b9256a717ab9 admin admin] PATH_INFO: `/v3/users/5cb0dae982a44ca39f12e659e42bec8f` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:26:11.953707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fb602-24f0-4131-be9e-b9256a717ab9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cb0dae982a44ca39f12e659e42bec8f)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:26:11.955838 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fb602-24f0-4131-be9e-b9256a717ab9 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:26:11.956850 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b00b6-db31-4718-8ba1-b64c25f722b0 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:26:12.023655 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-045fb602-24f0-4131-be9e-b9256a717ab9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cb0dae982a44ca39f12e659e42bec8f'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:26:12.036327 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-045fb602-24f0-4131-be9e-b9256a717ab9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cb0dae982a44ca39f12e659e42bec8f'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:26:12.047366 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:26:12.053091 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-045fb602-24f0-4131-be9e-b9256a717ab9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cb0dae982a44ca39f12e659e42bec8f'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:26:12.054098 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-045fb602-24f0-4131-be9e-b9256a717ab9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cb0dae982a44ca39f12e659e42bec8f'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:26:12.061951 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd071c-f982-4461-b862-cba8436a86d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:26:12.062582 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd071c-f982-4461-b862-cba8436a86d1 service nova] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:26:12.062852 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd071c-f982-4461-b862-cba8436a86d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:26:12.063124 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd071c-f982-4461-b862-cba8436a86d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:26:12.065471 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-045fb602-24f0-4131-be9e-b9256a717ab9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cb0dae982a44ca39f12e659e42bec8f'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:26:12.068035 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-6acd071c-f982-4461-b862-cba8436a86d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:26:12.073472 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1467/2920] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:26:11 2022] DELETE /identity/v3/users/5cb0dae982a44ca39f12e659e42bec8f => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:26:12.106591 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=100 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:26:12.147399 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd071c-f982-4461-b862-cba8436a86d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:26:12.149201 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd071c-f982-4461-b862-cba8436a86d1 service nova] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:26:12.159249 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1454/2921] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:26:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 15:26:12.286241 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fb602-24f0-4131-be9e-b9256a717ab9 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:26:12.288052 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-045fb602-24f0-4131-be9e-b9256a717ab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:26:12.368905 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fb602-24f0-4131-be9e-b9256a717ab9 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:26:12.374434 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c19bd-722b-4a6d-b8db-aed00a6e5404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:26:12.375768 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c19bd-722b-4a6d-b8db-aed00a6e5404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:26:12.375768 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c19bd-722b-4a6d-b8db-aed00a6e5404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:26:12.376020 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c19bd-722b-4a6d-b8db-aed00a6e5404 admin admin] PATH_INFO: `/v3/projects/73f7913c6fa84758b4ce8455d14ef3fe` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:26:12.377010 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c19bd-722b-4a6d-b8db-aed00a6e5404 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73f7913c6fa84758b4ce8455d14ef3fe)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:26:12.379510 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c19bd-722b-4a6d-b8db-aed00a6e5404 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:26:12.438064 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-6b2c19bd-722b-4a6d-b8db-aed00a6e5404 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73f7913c6fa84758b4ce8455d14ef3fe', 'request_id': 'req-6b2c19bd-722b-4a6d-b8db-aed00a6e5404'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:26:12.448479 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-6b2c19bd-722b-4a6d-b8db-aed00a6e5404 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73f7913c6fa84758b4ce8455d14ef3fe', 'request_id': 'req-6b2c19bd-722b-4a6d-b8db-aed00a6e5404'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:26:12.458830 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-6b2c19bd-722b-4a6d-b8db-aed00a6e5404 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73f7913c6fa84758b4ce8455d14ef3fe', 'request_id': 'req-6b2c19bd-722b-4a6d-b8db-aed00a6e5404'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:26:12.475680 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-6b2c19bd-722b-4a6d-b8db-aed00a6e5404 admin admin] The token cache is being invalidate because project 73f7913c6fa84758b4ce8455d14ef3fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:26:12.475926 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-6b2c19bd-722b-4a6d-b8db-aed00a6e5404 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:26:12.477449 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-6b2c19bd-722b-4a6d-b8db-aed00a6e5404 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:26:12.479502 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1468/2922] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:26:12 2022] DELETE /identity/v3/projects/73f7913c6fa84758b4ce8455d14ef3fe => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:26:22.154688 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=15 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:26:22.471533 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=58,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:26:27.597606 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a24b9-62a3-4dff-9a15-2e151c3f3d02 None None] REQUEST_METHOD: `POST` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:26:27.598214 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a24b9-62a3-4dff-9a15-2e151c3f3d02 None None] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:26:27.598214 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a24b9-62a3-4dff-9a15-2e151c3f3d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:26:27.618242 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.auth.core [None req-701a24b9-62a3-4dff-9a15-2e151c3f3d02 None None] MFA Rules not processed for user `0a70540aee424f3896e43e4d6af5c63a`. Rule list: `[]` (Enabled: `True`). {{(pid=81684) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:26:27.619473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-701a24b9-62a3-4dff-9a15-2e151c3f3d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:26:27.679230 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1455/2923] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:26:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:26:28.030085 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c19bd-722b-4a6d-b8db-aed00a6e5404 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:26:28.032373 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-6b2c19bd-722b-4a6d-b8db-aed00a6e5404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:26:28.100351 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c19bd-722b-4a6d-b8db-aed00a6e5404 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:26:28.104408 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d67fd-5c98-4f13-8351-bdab02ebe15c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:26:28.105034 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d67fd-5c98-4f13-8351-bdab02ebe15c service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:26:28.105349 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d67fd-5c98-4f13-8351-bdab02ebe15c service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:26:28.105652 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d67fd-5c98-4f13-8351-bdab02ebe15c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:26:28.108104 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-694d67fd-5c98-4f13-8351-bdab02ebe15c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:26:28.118949 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d67fd-5c98-4f13-8351-bdab02ebe15c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:26:28.120405 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d67fd-5c98-4f13-8351-bdab02ebe15c service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:26:28.129036 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1469/2924] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:26:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:26:37.672226 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=25 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:26:38.125165 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=26 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:26:44.309491 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a24b9-62a3-4dff-9a15-2e151c3f3d02 None None] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:26:44.324873 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a24b9-62a3-4dff-9a15-2e151c3f3d02 None None] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:26:44.329165 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f19f90-6d60-4436-ba41-4811c3281bbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:26:44.329778 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f19f90-6d60-4436-ba41-4811c3281bbc service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:26:44.330053 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f19f90-6d60-4436-ba41-4811c3281bbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:26:44.330305 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f19f90-6d60-4436-ba41-4811c3281bbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:26:44.333293 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-06f19f90-6d60-4436-ba41-4811c3281bbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:26:44.402718 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f19f90-6d60-4436-ba41-4811c3281bbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:26:44.404155 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f19f90-6d60-4436-ba41-4811c3281bbc service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:26:44.413712 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1456/2925] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:26:44 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:26:54.409602 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=30 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:28:44.299348 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0b19d-1674-4908-bd80-4bcaa80a04c4 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:28:44.299913 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0b19d-1674-4908-bd80-4bcaa80a04c4 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:28:44.299913 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0b19d-1674-4908-bd80-4bcaa80a04c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:28:44.321272 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-75f0b19d-1674-4908-bd80-4bcaa80a04c4 None None] MFA Rules not processed for user `0a70540aee424f3896e43e4d6af5c63a`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:28:44.322504 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-75f0b19d-1674-4908-bd80-4bcaa80a04c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:28:44.329988 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1470/2926] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:28:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:28:50.972433 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f19f90-6d60-4436-ba41-4811c3281bbc service neutron] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:28:50.973988 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-06f19f90-6d60-4436-ba41-4811c3281bbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:28:50.985440 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f19f90-6d60-4436-ba41-4811c3281bbc service neutron] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:28:50.989638 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91655125-f64b-4f6e-8e91-32514e2edbbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:28:50.990261 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91655125-f64b-4f6e-8e91-32514e2edbbe service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:28:50.990532 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91655125-f64b-4f6e-8e91-32514e2edbbe service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:28:50.990795 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91655125-f64b-4f6e-8e91-32514e2edbbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:28:50.992583 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-91655125-f64b-4f6e-8e91-32514e2edbbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:28:51.003198 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91655125-f64b-4f6e-8e91-32514e2edbbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:28:51.004711 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91655125-f64b-4f6e-8e91-32514e2edbbe service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:28:51.014154 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1457/2927] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:28:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:28:54.315575 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=3 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:29:01.011131 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=6 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:29:02.484788 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0b19d-1674-4908-bd80-4bcaa80a04c4 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:29:02.495792 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0b19d-1674-4908-bd80-4bcaa80a04c4 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:29:02.499683 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6502494-40d3-4b25-a27e-3aa54645a008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:29:02.500267 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6502494-40d3-4b25-a27e-3aa54645a008 service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:29:02.500543 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6502494-40d3-4b25-a27e-3aa54645a008 service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:29:02.500932 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6502494-40d3-4b25-a27e-3aa54645a008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:29:02.502863 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-e6502494-40d3-4b25-a27e-3aa54645a008 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:29:02.579080 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6502494-40d3-4b25-a27e-3aa54645a008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:29:02.580515 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6502494-40d3-4b25-a27e-3aa54645a008 service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:29:02.591787 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1471/2928] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:29:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:29:03.791046 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91655125-f64b-4f6e-8e91-32514e2edbbe service neutron] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:29:03.800488 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91655125-f64b-4f6e-8e91-32514e2edbbe service neutron] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:29:03.804528 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86deeb-bc2f-4e92-b7e8-ef2d768bebbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:29:03.805115 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86deeb-bc2f-4e92-b7e8-ef2d768bebbd service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:29:03.805422 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86deeb-bc2f-4e92-b7e8-ef2d768bebbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:29:03.805690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86deeb-bc2f-4e92-b7e8-ef2d768bebbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:29:03.807425 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-cb86deeb-bc2f-4e92-b7e8-ef2d768bebbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:29:03.822457 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86deeb-bc2f-4e92-b7e8-ef2d768bebbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:29:03.823895 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86deeb-bc2f-4e92-b7e8-ef2d768bebbd service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:29:03.835594 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1458/2929] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:29:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:29:12.586874 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=34 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:29:13.830168 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=6 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:30:43.460311 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917729c3-3be8-43ea-ba94-8c0314b620d3 None None] REQUEST_METHOD: `POST` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:30:43.460899 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917729c3-3be8-43ea-ba94-8c0314b620d3 None None] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:30:43.460899 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917729c3-3be8-43ea-ba94-8c0314b620d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:30:43.492823 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.auth.core [None req-917729c3-3be8-43ea-ba94-8c0314b620d3 None None] MFA Rules not processed for user `0a70540aee424f3896e43e4d6af5c63a`. Rule list: `[]` (Enabled: `True`). {{(pid=81683) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 15:30:43.496167 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-917729c3-3be8-43ea-ba94-8c0314b620d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:30:43.523707 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1472/2930] 10.222.0.44 () {58 vars in 958 bytes} [Sun Sep 11 15:30:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 15:30:43.867860 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86deeb-bc2f-4e92-b7e8-ef2d768bebbd service neutron] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:30:43.888191 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86deeb-bc2f-4e92-b7e8-ef2d768bebbd service neutron] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:30:43.892707 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5336d92-2ed1-4bb5-921d-2f936bbff17c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:30:43.893361 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5336d92-2ed1-4bb5-921d-2f936bbff17c service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:30:43.893636 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5336d92-2ed1-4bb5-921d-2f936bbff17c service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:30:43.893907 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5336d92-2ed1-4bb5-921d-2f936bbff17c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:30:43.896384 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b5336d92-2ed1-4bb5-921d-2f936bbff17c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:30:43.919624 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5336d92-2ed1-4bb5-921d-2f936bbff17c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:30:43.921081 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5336d92-2ed1-4bb5-921d-2f936bbff17c service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:30:43.929653 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1459/2931] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:30:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:30:44.411124 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917729c3-3be8-43ea-ba94-8c0314b620d3 None None] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:30:44.412660 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-917729c3-3be8-43ea-ba94-8c0314b620d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:30:44.500232 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917729c3-3be8-43ea-ba94-8c0314b620d3 None None] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:30:44.505397 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94ca6c-afb2-4933-b129-62721a213f8a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead06362f8a2444bbfe1405cd602f1c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:30:44.506302 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94ca6c-afb2-4933-b129-62721a213f8a service placement] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:30:44.506302 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94ca6c-afb2-4933-b129-62721a213f8a service placement] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:30:44.507533 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94ca6c-afb2-4933-b129-62721a213f8a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:30:44.509522 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-ff94ca6c-afb2-4933-b129-62721a213f8a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:30:44.521819 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94ca6c-afb2-4933-b129-62721a213f8a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:30:44.523419 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94ca6c-afb2-4933-b129-62721a213f8a service placement] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:30:44.533464 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1473/2932] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:30:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:30:47.118690 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5336d92-2ed1-4bb5-921d-2f936bbff17c service neutron] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:30:47.120261 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b5336d92-2ed1-4bb5-921d-2f936bbff17c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:30:47.145638 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5336d92-2ed1-4bb5-921d-2f936bbff17c service neutron] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:30:47.152800 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349723f-70ac-4927-b576-81d4b130a42a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:30:47.153700 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349723f-70ac-4927-b576-81d4b130a42a service nova] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:30:47.154019 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349723f-70ac-4927-b576-81d4b130a42a service nova] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:30:47.154442 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349723f-70ac-4927-b576-81d4b130a42a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:30:47.157094 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-0349723f-70ac-4927-b576-81d4b130a42a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:30:47.174228 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349723f-70ac-4927-b576-81d4b130a42a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:30:47.175931 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349723f-70ac-4927-b576-81d4b130a42a service nova] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:30:47.187633 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1460/2933] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:30:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:30:54.529236 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=50 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:30:56.863133 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94ca6c-afb2-4933-b129-62721a213f8a service placement] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:30:56.865338 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-ff94ca6c-afb2-4933-b129-62721a213f8a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:30:56.888067 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94ca6c-afb2-4933-b129-62721a213f8a service placement] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:30:56.895198 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83759247-a632-4de3-a584-d19734f1c8f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:30:56.895887 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83759247-a632-4de3-a584-d19734f1c8f5 service nova] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:30:56.896165 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83759247-a632-4de3-a584-d19734f1c8f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:30:56.896441 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83759247-a632-4de3-a584-d19734f1c8f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:30:56.898320 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-83759247-a632-4de3-a584-d19734f1c8f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:30:56.911379 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83759247-a632-4de3-a584-d19734f1c8f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:30:56.912959 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83759247-a632-4de3-a584-d19734f1c8f5 service nova] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:30:56.926430 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1474/2934] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:30:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:30:57.185038 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=24 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:31:00.589410 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349723f-70ac-4927-b576-81d4b130a42a service nova] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:31:00.600953 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349723f-70ac-4927-b576-81d4b130a42a service nova] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:31:00.607297 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f70221-270c-460b-b00b-9fce2b7b796d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '453297dac64046d19fc1c331d2d85abc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:31:00.607905 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f70221-270c-460b-b00b-9fce2b7b796d service nova] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:31:00.608174 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f70221-270c-460b-b00b-9fce2b7b796d service nova] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:31:00.608434 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f70221-270c-460b-b00b-9fce2b7b796d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:31:00.610323 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-87f70221-270c-460b-b00b-9fce2b7b796d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:31:00.623692 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f70221-270c-460b-b00b-9fce2b7b796d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:31:00.625334 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f70221-270c-460b-b00b-9fce2b7b796d service nova] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:31:00.636939 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1461/2935] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:31:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:31:06.920041 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=6 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:31:10.631925 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=6 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:31:46.883076 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83759247-a632-4de3-a584-d19734f1c8f5 service nova] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:31:46.884670 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-83759247-a632-4de3-a584-d19734f1c8f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:31:46.923434 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83759247-a632-4de3-a584-d19734f1c8f5 service nova] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:31:46.929229 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e54ca-3bc3-4c4b-8d10-2fc7d8d3ca9c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead06362f8a2444bbfe1405cd602f1c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:31:46.929833 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e54ca-3bc3-4c4b-8d10-2fc7d8d3ca9c service placement] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:31:46.930098 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e54ca-3bc3-4c4b-8d10-2fc7d8d3ca9c service placement] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:31:46.930382 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e54ca-3bc3-4c4b-8d10-2fc7d8d3ca9c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:31:46.932314 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-664e54ca-3bc3-4c4b-8d10-2fc7d8d3ca9c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:31:46.944526 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664e54ca-3bc3-4c4b-8d10-2fc7d8d3ca9c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:31:46.946114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664e54ca-3bc3-4c4b-8d10-2fc7d8d3ca9c service placement] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:31:46.956483 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1475/2936] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:31:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:31:56.952235 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=15 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:32:28.880446 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f70221-270c-460b-b00b-9fce2b7b796d service nova] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:32:28.901208 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f70221-270c-460b-b00b-9fce2b7b796d service nova] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:32:28.905651 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c45da-e284-43d3-b7c7-f40520ab7825 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:32:28.906228 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c45da-e284-43d3-b7c7-f40520ab7825 service neutron] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:32:28.906528 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c45da-e284-43d3-b7c7-f40520ab7825 service neutron] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:32:28.906797 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c45da-e284-43d3-b7c7-f40520ab7825 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:32:28.908760 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-423c45da-e284-43d3-b7c7-f40520ab7825 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:32:28.921087 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c45da-e284-43d3-b7c7-f40520ab7825 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:32:28.922533 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c45da-e284-43d3-b7c7-f40520ab7825 service neutron] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:32:28.931613 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1462/2937] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:32:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:32:38.927578 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=10 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:32:42.461935 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e54ca-3bc3-4c4b-8d10-2fc7d8d3ca9c service placement] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:32:42.471761 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e54ca-3bc3-4c4b-8d10-2fc7d8d3ca9c service placement] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:32:42.475427 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b806d9-3d6e-49b0-b874-16ab93dd40fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '697b73f182e6419a84e0c8c6b0fc0c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:32:42.476057 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b806d9-3d6e-49b0-b874-16ab93dd40fa service neutron] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:32:42.476327 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b806d9-3d6e-49b0-b874-16ab93dd40fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:32:42.476595 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b806d9-3d6e-49b0-b874-16ab93dd40fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:32:42.478505 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-e9b806d9-3d6e-49b0-b874-16ab93dd40fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:32:42.567361 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b806d9-3d6e-49b0-b874-16ab93dd40fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:32:42.568799 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b806d9-3d6e-49b0-b874-16ab93dd40fa service neutron] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:32:42.588551 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1476/2938] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:32:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:32:45.500473 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c45da-e284-43d3-b7c7-f40520ab7825 service neutron] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:32:45.514508 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c45da-e284-43d3-b7c7-f40520ab7825 service neutron] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:32:45.526353 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f1990-e95e-4e56-95be-050a56b33071 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead06362f8a2444bbfe1405cd602f1c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:32:45.527193 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f1990-e95e-4e56-95be-050a56b33071 service placement] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:32:45.527612 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f1990-e95e-4e56-95be-050a56b33071 service placement] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:32:45.527915 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f1990-e95e-4e56-95be-050a56b33071 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:32:45.529892 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b59f1990-e95e-4e56-95be-050a56b33071 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:32:45.545750 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59f1990-e95e-4e56-95be-050a56b33071 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:32:45.547277 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59f1990-e95e-4e56-95be-050a56b33071 service placement] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:32:45.577155 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1463/2939] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:32:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:32:48.742100 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b806d9-3d6e-49b0-b874-16ab93dd40fa service neutron] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:32:48.755785 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b806d9-3d6e-49b0-b874-16ab93dd40fa service neutron] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:32:48.764388 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4603fc3e-aff0-458c-b365-dffb81782f8b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead06362f8a2444bbfe1405cd602f1c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:32:48.765317 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603fc3e-aff0-458c-b365-dffb81782f8b service placement] REQUEST_METHOD: `GET` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:32:48.765786 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603fc3e-aff0-458c-b365-dffb81782f8b service placement] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:32:48.766223 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603fc3e-aff0-458c-b365-dffb81782f8b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:32:48.769044 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-4603fc3e-aff0-458c-b365-dffb81782f8b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:32:48.786600 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4603fc3e-aff0-458c-b365-dffb81782f8b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:32:48.788349 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4603fc3e-aff0-458c-b365-dffb81782f8b service placement] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:32:48.800852 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1477/2940] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:32:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:32:48.831482 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f1990-e95e-4e56-95be-050a56b33071 service placement] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:32:48.833662 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-b59f1990-e95e-4e56-95be-050a56b33071 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:32:48.849538 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f1990-e95e-4e56-95be-050a56b33071 service placement] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:32:48.855593 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee5ab32-29cb-4cff-b33c-7dd2787dccfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:32:48.856485 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee5ab32-29cb-4cff-b33c-7dd2787dccfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:32:48.856822 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee5ab32-29cb-4cff-b33c-7dd2787dccfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:32:48.857172 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee5ab32-29cb-4cff-b33c-7dd2787dccfa admin admin] PATH_INFO: `/v3/users/0a70540aee424f3896e43e4d6af5c63a` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:32:48.858567 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee5ab32-29cb-4cff-b33c-7dd2787dccfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a70540aee424f3896e43e4d6af5c63a)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:32:48.860627 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee5ab32-29cb-4cff-b33c-7dd2787dccfa admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:32:48.917715 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8ee5ab32-29cb-4cff-b33c-7dd2787dccfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a70540aee424f3896e43e4d6af5c63a'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:32:48.926083 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8ee5ab32-29cb-4cff-b33c-7dd2787dccfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a70540aee424f3896e43e4d6af5c63a'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:32:48.938553 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8ee5ab32-29cb-4cff-b33c-7dd2787dccfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a70540aee424f3896e43e4d6af5c63a'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:32:48.939204 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8ee5ab32-29cb-4cff-b33c-7dd2787dccfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a70540aee424f3896e43e4d6af5c63a'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:32:48.946450 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-8ee5ab32-29cb-4cff-b33c-7dd2787dccfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a70540aee424f3896e43e4d6af5c63a'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:32:48.952190 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1464/2941] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:32:48 2022] DELETE /identity/v3/users/0a70540aee424f3896e43e4d6af5c63a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:32:49.137440 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4603fc3e-aff0-458c-b365-dffb81782f8b service placement] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:32:49.150424 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4603fc3e-aff0-458c-b365-dffb81782f8b service placement] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:32:49.212691 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1ab21-29ff-4fca-8b0c-43bc01dee225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:32:49.213773 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1ab21-29ff-4fca-8b0c-43bc01dee225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:32:49.214081 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1ab21-29ff-4fca-8b0c-43bc01dee225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:32:49.214512 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1ab21-29ff-4fca-8b0c-43bc01dee225 admin admin] PATH_INFO: `/v3/projects/521053acc38643e09eb22ead90cd2ba1` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:32:49.216162 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f1ab21-29ff-4fca-8b0c-43bc01dee225 admin admin] RBAC: Authorizing `identity:delete_project(project_id=521053acc38643e09eb22ead90cd2ba1)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:32:49.218134 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f1ab21-29ff-4fca-8b0c-43bc01dee225 admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:32:49.274780 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-47f1ab21-29ff-4fca-8b0c-43bc01dee225 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '521053acc38643e09eb22ead90cd2ba1', 'request_id': 'req-47f1ab21-29ff-4fca-8b0c-43bc01dee225'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:32:49.279213 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:32:49.282114 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-47f1ab21-29ff-4fca-8b0c-43bc01dee225 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '521053acc38643e09eb22ead90cd2ba1', 'request_id': 'req-47f1ab21-29ff-4fca-8b0c-43bc01dee225'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:32:49.289678 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-47f1ab21-29ff-4fca-8b0c-43bc01dee225 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '521053acc38643e09eb22ead90cd2ba1', 'request_id': 'req-47f1ab21-29ff-4fca-8b0c-43bc01dee225'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:32:49.303220 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-47f1ab21-29ff-4fca-8b0c-43bc01dee225 admin admin] The token cache is being invalidate because project 521053acc38643e09eb22ead90cd2ba1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:32:49.303500 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-47f1ab21-29ff-4fca-8b0c-43bc01dee225 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:32:49.304128 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-47f1ab21-29ff-4fca-8b0c-43bc01dee225 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:32:49.305660 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1478/2942] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:32:49 2022] DELETE /identity/v3/projects/521053acc38643e09eb22ead90cd2ba1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:32:49.310138 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee5ab32-29cb-4cff-b33c-7dd2787dccfa admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:32:49.311652 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-8ee5ab32-29cb-4cff-b33c-7dd2787dccfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:32:49.325900 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee5ab32-29cb-4cff-b33c-7dd2787dccfa admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:32:49.390166 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9b717-9cae-4835-8ec4-7e5ee53a96b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:32:49.390806 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9b717-9cae-4835-8ec4-7e5ee53a96b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:32:49.391105 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9b717-9cae-4835-8ec4-7e5ee53a96b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:32:49.391370 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9b717-9cae-4835-8ec4-7e5ee53a96b3 admin admin] PATH_INFO: `/v3/users/9689c4018b334a9380f269772758e2f5` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:32:49.392644 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e9b717-9cae-4835-8ec4-7e5ee53a96b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9689c4018b334a9380f269772758e2f5)` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:32:49.394717 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e9b717-9cae-4835-8ec4-7e5ee53a96b3 admin admin] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:32:49.450803 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-72e9b717-9cae-4835-8ec4-7e5ee53a96b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9689c4018b334a9380f269772758e2f5'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:32:49.459438 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-72e9b717-9cae-4835-8ec4-7e5ee53a96b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9689c4018b334a9380f269772758e2f5'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:32:49.464750 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:32:49.473327 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-72e9b717-9cae-4835-8ec4-7e5ee53a96b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9689c4018b334a9380f269772758e2f5'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:32:49.474141 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-72e9b717-9cae-4835-8ec4-7e5ee53a96b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9689c4018b334a9380f269772758e2f5'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:32:49.481173 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.notifications [None req-72e9b717-9cae-4835-8ec4-7e5ee53a96b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9689c4018b334a9380f269772758e2f5'} {{(pid=81684) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:32:49.487418 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1465/2943] 10.222.0.44 () {58 vars in 1224 bytes} [Sun Sep 11 15:32:49 2022] DELETE /identity/v3/users/9689c4018b334a9380f269772758e2f5 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:32:49.667604 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1ab21-29ff-4fca-8b0c-43bc01dee225 admin admin] Authenticating user token {{(pid=81683) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:32:49.669619 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.fernet_utils [None req-47f1ab21-29ff-4fca-8b0c-43bc01dee225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81683) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:32:49.684323 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1ab21-29ff-4fca-8b0c-43bc01dee225 admin admin] Validating token access rules against request {{(pid=81683) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:32:49.744214 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f341c36-c625-43a2-bfc1-1ec62239d02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6c6442222c54f34a84fdb4577045a9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '866dbf4792e0480eb3f18a3e97bbaa75', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81683) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:32:49.744940 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f341c36-c625-43a2-bfc1-1ec62239d02a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:32:49.745102 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f341c36-c625-43a2-bfc1-1ec62239d02a admin admin] SCRIPT_NAME: `/identity` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:32:49.745412 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f341c36-c625-43a2-bfc1-1ec62239d02a admin admin] PATH_INFO: `/v3/projects/e89b26aada41478396b13c82a270ab56` {{(pid=81683) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:32:49.746593 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f341c36-c625-43a2-bfc1-1ec62239d02a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e89b26aada41478396b13c82a270ab56)` {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:32:49.748492 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f341c36-c625-43a2-bfc1-1ec62239d02a admin admin] RBAC: Authorization granted {{(pid=81683) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:32:49.795731 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5f341c36-c625-43a2-bfc1-1ec62239d02a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e89b26aada41478396b13c82a270ab56', 'request_id': 'req-5f341c36-c625-43a2-bfc1-1ec62239d02a'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:32:49.801300 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5f341c36-c625-43a2-bfc1-1ec62239d02a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e89b26aada41478396b13c82a270ab56', 'request_id': 'req-5f341c36-c625-43a2-bfc1-1ec62239d02a'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:32:49.807788 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5f341c36-c625-43a2-bfc1-1ec62239d02a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e89b26aada41478396b13c82a270ab56', 'request_id': 'req-5f341c36-c625-43a2-bfc1-1ec62239d02a'} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:32:49.821904 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5f341c36-c625-43a2-bfc1-1ec62239d02a admin admin] The token cache is being invalidate because project e89b26aada41478396b13c82a270ab56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81683) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 15:32:49.822190 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5f341c36-c625-43a2-bfc1-1ec62239d02a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:32:49.822825 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG keystone.notifications [None req-5f341c36-c625-43a2-bfc1-1ec62239d02a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81683) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 15:32:49.824353 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: [pid: 81683|app: 0|req: 1479/2944] 10.222.0.44 () {58 vars in 1233 bytes} [Sun Sep 11 15:32:49 2022] DELETE /identity/v3/projects/e89b26aada41478396b13c82a270ab56 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 15:32:59.483621 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=9,keystone:DELETE=1 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:32:59.818894 dv-cinder-839768-7-iscsi devstack@keystone.service[81683]: DEBUG dbcounter [-] [81683] Writing DB stats keystone:DELETE=9,keystone:SELECT=56,keystone:INSERT=2 {{(pid=81683) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 15:36:51.873828 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9b717-9cae-4835-8ec4-7e5ee53a96b3 admin admin] Authenticating user token {{(pid=81684) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 15:36:51.876422 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-72e9b717-9cae-4835-8ec4-7e5ee53a96b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:36:51.966006 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9b717-9cae-4835-8ec4-7e5ee53a96b3 admin admin] Validating token access rules against request {{(pid=81684) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 15:36:52.029281 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe2e3f-0bab-4a0d-82a1-db709168ce02 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead06362f8a2444bbfe1405cd602f1c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '70ac515938a24c45ae6302dbac46a1ab', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81684) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 15:36:52.029895 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe2e3f-0bab-4a0d-82a1-db709168ce02 service placement] REQUEST_METHOD: `GET` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 15:36:52.030174 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe2e3f-0bab-4a0d-82a1-db709168ce02 service placement] SCRIPT_NAME: `/identity` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 15:36:52.030440 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe2e3f-0bab-4a0d-82a1-db709168ce02 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81684) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 15:36:52.032351 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.fernet_utils [None req-75fe2e3f-0bab-4a0d-82a1-db709168ce02 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81684) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 15:36:52.044195 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fe2e3f-0bab-4a0d-82a1-db709168ce02 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 15:36:52.045845 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fe2e3f-0bab-4a0d-82a1-db709168ce02 service placement] RBAC: Authorization granted {{(pid=81684) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 15:36:52.056695 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: [pid: 81684|app: 0|req: 1466/2945] 10.222.0.44 () {60 vars in 1365 bytes} [Sun Sep 11 15:36:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 15:37:02.052089 dv-cinder-839768-7-iscsi devstack@keystone.service[81684]: DEBUG dbcounter [-] [81684] Writing DB stats keystone:SELECT=58 {{(pid=81684) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}